Jul 24 03:17:24.923767 ip-10-0-140-227 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 03:17:25.299691 ip-10-0-140-227 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 03:17:25.299691 ip-10-0-140-227 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 03:17:25.299691 ip-10-0-140-227 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 03:17:25.299691 ip-10-0-140-227 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 03:17:25.299691 ip-10-0-140-227 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 03:17:25.301677 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.301585    2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 03:17:25.307291 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307271    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 03:17:25.307291 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307291    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307296    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307300    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307303    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307306    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307308    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307312    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307315    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307318    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307321    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307324    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307328    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307330    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307333    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307336    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307338    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307341    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307343    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307346    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307349    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 03:17:25.307365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307351    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307354    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307356    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307359    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307361    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307364    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307368    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307370    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307373    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307376    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307380    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307383    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307385    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307388    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307390    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307393    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307395    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307398    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307400    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307403    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 03:17:25.307882 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307405    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307408    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307410    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307412    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307415    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307417    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307420    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307436    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307440    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307444    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307448    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307453    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307458    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307463    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307467    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307469    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307472    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307475    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307477    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 03:17:25.308361 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307480    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307482    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307485    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307487    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307491    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307494    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307497    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307499    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307502    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307504    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307507    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307510    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307512    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307515    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307520    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307524    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307527    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307530    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307533    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 03:17:25.308851 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307536    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307539    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307541    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307544    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307547    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307549    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307554    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307558    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307561    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307564    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307566    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307569    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307571    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307574    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307576    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307579    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307581    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307584    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307587    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.307589    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 03:17:25.309467 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308328    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308335    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308338    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308340    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308343    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308346    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308348    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308351    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308353    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308356    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308358    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308361    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308364    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308367    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308369    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308372    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308375    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308378    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308381    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 03:17:25.310162 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308384    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308386    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308389    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308391    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308394    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308396    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308399    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308401    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308404    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308407    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308409    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308411    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308415    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308417    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308419    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308437    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308440    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308443    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308445    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308448    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 03:17:25.310744 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308451    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308455    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308459    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308462    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308464    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308467    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308469    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308473    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308476    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308479    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308482    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308486    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308490    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308493    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308496    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308498    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308501    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308503    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308506    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 03:17:25.311258 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308509    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308512    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308514    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308517    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308519    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308521    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308525    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308527    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308530    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308533    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308535    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308539    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308541    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308543    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308546    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308548    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308551    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308553    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308556    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308558    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 03:17:25.311753 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308561    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308564    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308567    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308570    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308573    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308576    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308578    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308581    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308584    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308586    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308588    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308591    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308593    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308595    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308598    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308600    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308603    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308605    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308608    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308611    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 03:17:25.312245 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.308613    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310823    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310835    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310839    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310842    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310845    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310851    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310856    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310861    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310864    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310868    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310871    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310874    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310877    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310880    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310883    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310886    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310889    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310892    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 03:17:25.312747 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310895    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310898    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310900    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310903    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310906    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310908    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310911    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310913    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310916    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310918    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310921    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310924    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310926    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310935    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310938    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310940    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310943    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310945    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310948    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310952    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 03:17:25.313215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310955    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310959    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310961    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310964    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310967    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310969    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310972    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310974    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310977    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310980    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310983    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310985    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310991    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310993    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310996    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.310999    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311001    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311004    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311007    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311009    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 03:17:25.313811 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311012    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311014    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311034    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311038    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311041    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311044    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311047    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311050    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311052    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311055    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311058    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311060    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311063    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311066    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311069    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311072    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311074    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311076    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311079    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311081    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 03:17:25.314300 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311084    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311086    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311089    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311091    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311094    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311097    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311100    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311102    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311105    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311107    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311109    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311112    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311114    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311117    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311119    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311122    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311126    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311129    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311133    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 03:17:25.314833 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311136    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311140    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311217    2574 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311225    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311233    2574 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311237    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311242    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311246    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311251    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311256    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311259    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311262    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311265    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311269    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311273    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311276    2574 flags.go:64] FLAG: --cgroup-root=""
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311279    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311281    2574 flags.go:64] FLAG: --client-ca-file=""
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311284    2574 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311288    2574 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311292    2574 flags.go:64] FLAG: --cluster-domain=""
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311295    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311298    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 03:17:25.315302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311301    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311304    2574 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311308    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311311    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311314    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311318    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311321    2574 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311324    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311327    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311330    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311333    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311337    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311340    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311343    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311346    2574 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311349    2574 flags.go:64] FLAG: --enable-server="true"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311352    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311357    2574 flags.go:64] FLAG: --event-burst="100"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311360    2574 flags.go:64] FLAG: --event-qps="50"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311362    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311365    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311368    2574 flags.go:64] FLAG: --eviction-hard=""
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311372    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311375    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311377    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 03:17:25.315900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311380    2574 flags.go:64] FLAG: --eviction-soft=""
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311383    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311386    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311388    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311392    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311394    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311397    2574 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311400    2574 flags.go:64] FLAG: --feature-gates=""
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311407    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311410    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311413    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311415    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311418    2574 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311436    2574 flags.go:64] FLAG: --help="false"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311440    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311445    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311449    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311454    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311458    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311462    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311465    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311467    2574 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311471    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 03:17:25.316508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311474    2574 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311476    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311479    2574 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311482    2574 flags.go:64] FLAG: --kube-reserved=""
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311485    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311488    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311491    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311493    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311496    2574 flags.go:64] FLAG: --lock-file=""
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311499    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311502    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311504    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311510    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311512    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311516    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311519    2574 flags.go:64] FLAG: --logging-format="text"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311522    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311525    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311528    2574 flags.go:64] FLAG: --manifest-url=""
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311531    2574 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311535    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311538    2574 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311543    2574 flags.go:64] FLAG: --max-pods="110"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311546    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311549    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 03:17:25.317088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311551    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311554    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311557    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311560    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311564    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311572    2574 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311576    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311578    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311581    2574 flags.go:64] FLAG: --pod-cidr=""
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311585    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311590    2574 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311593    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311596    2574 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311599    2574 flags.go:64] FLAG: --port="10250"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311602    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311605    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06f364e0904b342a1"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311608    2574 flags.go:64] FLAG: --qos-reserved=""
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311611    2574 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311614    2574 flags.go:64] FLAG: --register-node="true"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311617    2574 flags.go:64] FLAG: --register-with-taints=""
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311620    2574 flags.go:64] FLAG: --registry-burst="10"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311623    2574 flags.go:64] FLAG: --registry-qps="5"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311626    2574 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311629    2574 flags.go:64] FLAG: --reserved-memory=""
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311633    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 03:17:25.317731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311635    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311638    2574 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311641    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311644    2574 flags.go:64] FLAG: --runonce="false"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311648    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311651    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311654    2574 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311656    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311659    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311662    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311665    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311669    2574 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311672    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311675    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311678    2574 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311680    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311683    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311686    2574 flags.go:64] FLAG: --system-cgroups=""
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311689    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311704    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311707    2574 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311709    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311714    2574 flags.go:64] FLAG: --tls-min-version=""
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311716    2574 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311719    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 03:17:25.318407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311722    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311725    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311728    2574 flags.go:64] FLAG: --v="2"
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311733    2574 flags.go:64] FLAG: --version="false"
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311737    2574 flags.go:64] FLAG: --vmodule=""
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311741    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.311744    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311862    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311866    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311869    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311874    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311878    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311881    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311884    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311888    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311890    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311893    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311895    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311900    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311902    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311905    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 03:17:25.319165 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311907    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311910    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311912    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311915    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311918    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311920    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311922    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311925    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311928    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311930    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311934    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311938    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311941    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311944    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311947    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311949    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311952    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311955    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311957    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311960    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 03:17:25.319714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311962    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311966    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311968    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311971    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311973    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311976    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311978    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311981    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311983    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311987    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311989    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311992    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311994    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311997    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.311999    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312002    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312004    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312006    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312009    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312011    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 03:17:25.320207 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312014    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312016    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312019    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312021    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312023    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312026    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312028    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312031    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312033    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312036    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312039    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312042    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312044    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312048    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312051    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312053    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312056    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312058    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312061    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312063    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 03:17:25.320703 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312066    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312069    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312072    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312075    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312077    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312079    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312082    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312085    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312087    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312090    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312092    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312095    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312097    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312100    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312102    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312104    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312107    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312109    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312113    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 03:17:25.321178 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312116    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312119    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312122    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312125    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312128    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.312131    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.312858    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.319742    2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.319763    2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.319785    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.319791    2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319851    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319856    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319859    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319863    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 03:17:25.321661 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319865    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319868    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319871    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319874    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319877    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319880    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319882    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319887    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319891    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319895    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319899    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319901    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319904    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319906    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319909    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319911    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319914    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319916    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319919    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 03:17:25.322033 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319922    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319924    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319927    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319929    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319932    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319934    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319939    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319942    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319945    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319947    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319950    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319952    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319955    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319958    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319960    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319963    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319965    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319968    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319970    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319972    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 03:17:25.322511 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319975    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319977    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319980    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319983    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319985    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319988    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319990    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319993    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319995    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.319998    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320001    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320003    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320006    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320008    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320011    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320013    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320016    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320018    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320021    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 03:17:25.323057 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320024    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320027    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320030    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320034    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320037    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320039    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320042    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320044    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320047    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320049    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320051    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320054    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320056    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320059    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320061    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320063    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320066    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320069    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320071    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320074    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 03:17:25.323536 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320076    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320078    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320081    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320083    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320086    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320088    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320091    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320093    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320096    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320098    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320101    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320104    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320106    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320110    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320112    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320115    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 03:17:25.324153 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320117    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.320122    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320243    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320247    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320250    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320254    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320257    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320261    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320263    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320266    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320269    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320271    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320274    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320277    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 03:17:25.324567 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320279    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320282    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320284    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320286    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320289    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320291    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320294    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320296    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320298    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320301    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320304    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320306    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320309    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320311    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320314    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320317    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320320    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320322    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320325    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320327    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320330    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 03:17:25.324997 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320332    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320335    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320337    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320339    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320342    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320344    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320347    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320349    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320352    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320354    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320357    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320359    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320362    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320364    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320367    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320369    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320371    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320374    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320376    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320379    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 03:17:25.325557 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320381    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320383    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320387    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320390    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320394    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320397    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320399    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320402    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320407    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320410    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320413    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320415    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320418    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320421    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320436    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320439    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320442    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320444    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320449    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 03:17:25.326042 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320451    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320454    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320456    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320459    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320461    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320464    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320466    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320469    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320471    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320474    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320477    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320479    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320482    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320484    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320487    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320489    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320492    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320494    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320496    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 03:17:25.326635 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320499    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320501    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320504    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320506    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320509    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320512    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320514    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:25.320517    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.320522    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.320653    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.322528    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.323628    2574 server.go:1019] "Starting client certificate rotation"
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.323747    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.323794    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 03:17:25.327418 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.325920    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 03:17:25.327859 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.327113    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 03:17:25.336763 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.336745    2574 log.go:25] "Validated CRI v1 runtime API"
Jul 24 03:17:25.339928 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.339909    2574 log.go:25] "Validated CRI v1 image API"
Jul 24 03:17:25.340860 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.340841    2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 03:17:25.344407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.344346    2574 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d9ac09d9-64fb-46a0-92e1-7b1030ddf26b:/dev/nvme0n1p3 eb2fc09e-bbb7-4f64-b41a-843a6a867036:/dev/nvme0n1p4]
Jul 24 03:17:25.344407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.344370    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 03:17:25.350271 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.350150    2574 manager.go:216] Machine: {Timestamp:2026-07-24 03:17:25.34893777 +0000 UTC m=+0.312031048 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097611 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2dadada37b88f5d06593130f2119a7 SystemUUID:ec2dadad-a37b-88f5-d065-93130f2119a7 BootID:5112e2d3-5a28-45ab-90d5-26ad78ca1bfa Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ea:ef:08:bd:93 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ea:ef:08:bd:93 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:8d:31:1a:91:c0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 03:17:25.350271 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.350262    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 03:17:25.350416 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.350345    2574 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 03:17:25.354134 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.354107    2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 03:17:25.354268 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.354135    2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-227.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 03:17:25.354512 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.354486    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 03:17:25.354811 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.354800    2574 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 03:17:25.354849 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.354813    2574 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 03:17:25.354849 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.354820    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 03:17:25.354849 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.354834    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 03:17:25.355601 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.355591    2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 03:17:25.357214 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.357202    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 03:17:25.357524 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.357514    2574 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 03:17:25.357561 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.357550    2574 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 03:17:25.357589 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.357565    2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 03:17:25.357622 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.357588    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 03:17:25.357622 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.357601    2574 kubelet.go:398] "Adding apiserver pod source"
Jul 24 03:17:25.357622 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.357620    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 03:17:25.358044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.358032    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 03:17:25.358092 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.358052    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 03:17:25.360416 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.360398    2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 03:17:25.362104 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.362086    2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 03:17:25.362195 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.362125    2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 03:17:25.364008 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.363993    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 03:17:25.364090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.364018    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 03:17:25.364090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.364027    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 03:17:25.364090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.364034    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 03:17:25.364090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.364046    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 03:17:25.364090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.364053    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 03:17:25.364090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.364064    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 03:17:25.364090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.364072    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 03:17:25.364090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.364083    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 03:17:25.364090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.364093    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 03:17:25.365000 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.364990    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 03:17:25.365057 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.365002    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 03:17:25.366383 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.366373    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 03:17:25.366449 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.366386    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 03:17:25.375567 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.375550    2574 server.go:1268] "Started kubelet"
Jul 24 03:17:25.375962 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.375926    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 03:17:25.376524 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.376502    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-227.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 03:17:25.376699 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.376617    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 03:17:25.376805 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.376715    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 03:17:25.376751 ip-10-0-140-227 systemd[1]: Started Kubernetes Kubelet.
Jul 24 03:17:25.377458 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.377214    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-227.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 03:17:25.377458 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.377225    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 03:17:25.380476 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.380452    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pkpqf"
Jul 24 03:17:25.380687 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.380670    2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 03:17:25.381257 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.381238    2574 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 03:17:25.382656 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.382630    2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 03:17:25.384201 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.382152    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-227.ec2.internal.18c51b7bb0547227  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-227.ec2.internal,UID:ip-10-0-140-227.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-227.ec2.internal,},FirstTimestamp:2026-07-24 03:17:25.375521319 +0000 UTC m=+0.338614596,LastTimestamp:2026-07-24 03:17:25.375521319 +0000 UTC m=+0.338614596,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-227.ec2.internal,}"
Jul 24 03:17:25.386946 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.386927    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 03:17:25.387042 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.386928    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 03:17:25.387452 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.387093    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-227.ec2.internal\" not found"
Jul 24 03:17:25.387643 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.387616    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 03:17:25.387740 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.387687    2574 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 03:17:25.387740 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.387713    2574 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 03:17:25.387879 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.387864    2574 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 03:17:25.387879 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.387880    2574 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 03:17:25.388037 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.387911    2574 factory.go:55] Registering systemd factory
Jul 24 03:17:25.388037 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.387927    2574 factory.go:223] Registration of the systemd container factory successfully
Jul 24 03:17:25.388199 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.388179    2574 factory.go:153] Registering CRI-O factory
Jul 24 03:17:25.388199 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.388201    2574 factory.go:223] Registration of the crio container factory successfully
Jul 24 03:17:25.388348 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.388260    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 03:17:25.388348 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.388285    2574 factory.go:103] Registering Raw factory
Jul 24 03:17:25.388348 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.388303    2574 manager.go:1196] Started watching for new ooms in manager
Jul 24 03:17:25.388793 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.388777    2574 manager.go:349] Starting recovery of all containers
Jul 24 03:17:25.390284 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.390262    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pkpqf"
Jul 24 03:17:25.396944 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.396686    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 03:17:25.397038 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.396837    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 03:17:25.399822 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.399777    2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-227.ec2.internal\" not found" node="ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.399965 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.399951    2574 manager.go:354] Recovery completed
Jul 24 03:17:25.404828 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.404814    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:25.406680 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.406661    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:25.406768 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.406696    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:25.406768 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.406706    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:25.407322 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.407309    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 03:17:25.407322 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.407322    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:25.407478 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.407321    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 03:17:25.407478 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.407372    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 03:17:25.408682 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.408665    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:25.408750 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.408702    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:25.408750 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.408717    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:25.408815 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.408769    2574 policy_none.go:49] "None policy: Start"
Jul 24 03:17:25.408815 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.408783    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 03:17:25.408815 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.408794    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 03:17:25.409741 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.409730    2574 policy_none.go:47] "Start"
Jul 24 03:17:25.442899 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.442881    2574 manager.go:337] "Starting Device Plugin manager"
Jul 24 03:17:25.442983 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.442923    2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 03:17:25.442983 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.442933    2574 server.go:87] "Starting device plugin registration server"
Jul 24 03:17:25.443236 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.443223    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 03:17:25.443292 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.443239    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 03:17:25.443394 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.443376    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 03:17:25.443498 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.443485    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 03:17:25.443498 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.443498    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 03:17:25.444097 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.444075    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 03:17:25.444175 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.444132    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-227.ec2.internal\" not found"
Jul 24 03:17:25.472143 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.472118    2574 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2613 into the system container \"/system.slice\": <nil>"
Jul 24 03:17:25.543932 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.543899    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:25.545148 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.545130    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:25.545229 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.545164    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:25.545229 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.545179    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:25.545229 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.545213    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.552957 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.552901    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.552957 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.552929    2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-227.ec2.internal\": node \"ip-10-0-140-227.ec2.internal\" not found"
Jul 24 03:17:25.566387 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.566362    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 03:17:25.566547 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.566411    2574 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 03:17:25.566547 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.566450    2574 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 03:17:25.566547 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.566491    2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 03:17:25.567035 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.567018    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-227.ec2.internal\" not found"
Jul 24 03:17:25.568459 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.568421    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 03:17:25.667328 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.667278    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-227.ec2.internal\" not found"
Jul 24 03:17:25.667328 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.667303    2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-227.ec2.internal"]
Jul 24 03:17:25.667595 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.667383    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:25.668468 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.668450    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:25.668558 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.668487    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:25.668558 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.668501    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:25.669831 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.669818    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:25.669984 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.669970    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.670033 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.670001    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:25.671307 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.671292    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:25.671384 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.671326    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:25.671384 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.671336    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:25.671384 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.671342    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:25.671384 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.671354    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:25.671384 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.671367    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:25.672459 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.672443    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.672539 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.672469    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 03:17:25.673216 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.673199    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 03:17:25.673216 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.673227    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 03:17:25.673363 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.673238    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeHasSufficientPID"
Jul 24 03:17:25.689844 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.689824    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bceac0aa275a63572a2fe0294b42f48b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal\" (UID: \"bceac0aa275a63572a2fe0294b42f48b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.689933 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.689852    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bceac0aa275a63572a2fe0294b42f48b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal\" (UID: \"bceac0aa275a63572a2fe0294b42f48b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.689933 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.689870    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb9eba28ae5571b83e98bdb2f8bef7f3-config\") pod \"kube-apiserver-proxy-ip-10-0-140-227.ec2.internal\" (UID: \"cb9eba28ae5571b83e98bdb2f8bef7f3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.696117 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.696093    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-227.ec2.internal\" not found" node="ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.699994 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.699974    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-227.ec2.internal\" not found" node="ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.767936 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.767904    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-227.ec2.internal\" not found"
Jul 24 03:17:25.790289 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.790267    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb9eba28ae5571b83e98bdb2f8bef7f3-config\") pod \"kube-apiserver-proxy-ip-10-0-140-227.ec2.internal\" (UID: \"cb9eba28ae5571b83e98bdb2f8bef7f3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.790379 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.790297    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bceac0aa275a63572a2fe0294b42f48b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal\" (UID: \"bceac0aa275a63572a2fe0294b42f48b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.790379 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.790314    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bceac0aa275a63572a2fe0294b42f48b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal\" (UID: \"bceac0aa275a63572a2fe0294b42f48b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.790379 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.790353    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bceac0aa275a63572a2fe0294b42f48b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal\" (UID: \"bceac0aa275a63572a2fe0294b42f48b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.790531 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.790373    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb9eba28ae5571b83e98bdb2f8bef7f3-config\") pod \"kube-apiserver-proxy-ip-10-0-140-227.ec2.internal\" (UID: \"cb9eba28ae5571b83e98bdb2f8bef7f3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.790531 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.790398    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bceac0aa275a63572a2fe0294b42f48b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal\" (UID: \"bceac0aa275a63572a2fe0294b42f48b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:25.868851 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.868764    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-227.ec2.internal\" not found"
Jul 24 03:17:25.969291 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:25.969253    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-227.ec2.internal\" not found"
Jul 24 03:17:25.999880 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:25.999847    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:26.003482 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.003438    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:26.070171 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.070129    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-227.ec2.internal\" not found"
Jul 24 03:17:26.170772 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.170709    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-227.ec2.internal\" not found"
Jul 24 03:17:26.271249 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.271210    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-227.ec2.internal\" not found"
Jul 24 03:17:26.305212 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.305179    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 03:17:26.323098 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.323066    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 03:17:26.323245 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.323227    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 03:17:26.323301 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.323261    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 03:17:26.323301 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.323262    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 03:17:26.358433 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.358399    2574 apiserver.go:52] "Watching apiserver"
Jul 24 03:17:26.374908 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.374881    2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 03:17:26.375452 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.375414    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-r9n22","openshift-ovn-kubernetes/ovnkube-node-zgfsh","kube-system/global-pull-secret-syncer-6vl7h","kube-system/konnectivity-agent-nxrsc","openshift-cluster-node-tuning-operator/tuned-8fxrm","openshift-multus/network-metrics-daemon-mp65w","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8","openshift-dns/node-resolver-s7jjg","openshift-image-registry/node-ca-5j4wp","openshift-multus/multus-additional-cni-plugins-4wrj9","openshift-multus/multus-gx5vw","openshift-network-diagnostics/network-check-target-5g2fw"]
Jul 24 03:17:26.376745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.376724    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r9n22"
Jul 24 03:17:26.377815 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.377795    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.378923 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.378896    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 03:17:26.379028 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.378932    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 03:17:26.379090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.379024    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2smvh\""
Jul 24 03:17:26.380565 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.379238    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 03:17:26.380565 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.379729    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-96hlp\""
Jul 24 03:17:26.380565 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.379981    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 03:17:26.380565 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.380316    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 03:17:26.380795 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.380545    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 03:17:26.381284 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.381259    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 03:17:26.381874 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.381261    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 03:17:26.381874 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.381344    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 03:17:26.381874 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.381388    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:26.381874 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.381567    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:26.382501 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.382483    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nxrsc"
Jul 24 03:17:26.383566 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.383547    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.384303 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.384282    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 03:17:26.384385 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.384315    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 03:17:26.384385 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.384341    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nsnr7\""
Jul 24 03:17:26.384562 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.384549    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:26.384644 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.384626    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:26.385530 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.385496    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 03:17:26.385640 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.385540    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 03:17:26.385640 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.385596    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.385745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.385726    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7jrv2\""
Jul 24 03:17:26.386773 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.386757    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s7jjg"
Jul 24 03:17:26.387079 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.387065    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 03:17:26.387153 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.387074    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:26.387607 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.387583    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 03:17:26.387733 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.387607    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 03:17:26.387785 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.387728    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ctsbq\""
Jul 24 03:17:26.388013 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.387979    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 03:17:26.388013 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.388005    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5j4wp"
Jul 24 03:17:26.388688 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.388670    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 03:17:26.388806 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.388790    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6kzdt\""
Jul 24 03:17:26.388987 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.388974    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 03:17:26.389285 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.389272    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.389999 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.389984    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 03:17:26.390171 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.390155    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 03:17:26.390234 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.390161    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 03:17:26.390234 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.390186    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-964lf\""
Jul 24 03:17:26.390505 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.390493    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.391331 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.391312    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 03:17:26.391409 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.391375    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xnn7b\""
Jul 24 03:17:26.391409 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.391402    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 03:17:26.391713 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.391683    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:26.391713 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.391702    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 03:17:26.391835 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.391722    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 03:17:26.391835 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.391746    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 03:17:26.391835 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.391755    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:26.392322 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.392305    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 03:17:26.392388 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.392377    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-49r5p\""
Jul 24 03:17:26.393769 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.393750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/559deb09-e606-40bb-ba17-b957dca7b41a-tmp-dir\") pod \"node-resolver-s7jjg\" (UID: \"559deb09-e606-40bb-ba17-b957dca7b41a\") " pod="openshift-dns/node-resolver-s7jjg"
Jul 24 03:17:26.393860 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.393774    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-node-log\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.393860 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.393805    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/298d82e2-1299-49d7-a29c-1d323a2c595c-ovn-node-metrics-cert\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.393860 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.393837    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-run\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.394010 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.393869    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-sys-fs\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.394010 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.393906    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-kubelet\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.394010 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.393937    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/298d82e2-1299-49d7-a29c-1d323a2c595c-env-overrides\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.394010 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.393964    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-sysconfig\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.394010 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.393982    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-var-lib-kubelet\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.394010 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.393997    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/298d82e2-1299-49d7-a29c-1d323a2c595c-ovnkube-script-lib\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.394010 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394012    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-sysctl-d\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394045    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/559deb09-e606-40bb-ba17-b957dca7b41a-hosts-file\") pod \"node-resolver-s7jjg\" (UID: \"559deb09-e606-40bb-ba17-b957dca7b41a\") " pod="openshift-dns/node-resolver-s7jjg"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394067    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-kubelet-dir\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394082    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-sys\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394095    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-socket-dir\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394116    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlgbn\" (UniqueName: \"kubernetes.io/projected/7d87b726-552b-44e6-9c18-61b33f67d9eb-kube-api-access-dlgbn\") pod \"iptables-alerter-r9n22\" (UID: \"7d87b726-552b-44e6-9c18-61b33f67d9eb\") " pod="openshift-network-operator/iptables-alerter-r9n22"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394136    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-etc-openvswitch\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394151    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jfsj\" (UniqueName: \"kubernetes.io/projected/298d82e2-1299-49d7-a29c-1d323a2c595c-kube-api-access-8jfsj\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394172    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5618041c-75a6-4c3b-a584-519fad1a4b20-agent-certs\") pod \"konnectivity-agent-nxrsc\" (UID: \"5618041c-75a6-4c3b-a584-519fad1a4b20\") " pod="kube-system/konnectivity-agent-nxrsc"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394190    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-modprobe-d\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394215    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-systemd\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394235    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91d31d75-fb0f-44f5-94b5-129970db4fea-tmp\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394264    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc5rp\" (UniqueName: \"kubernetes.io/projected/91d31d75-fb0f-44f5-94b5-129970db4fea-kube-api-access-kc5rp\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394278    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-etc-selinux\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.394339 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394300    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-run-systemd\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394379    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-run-openvswitch\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394339    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 03:12:25 +0000 UTC" deadline="2028-03-08 01:02:11.70203888 +0000 UTC"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394400    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14229h44m45.307642514s"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394411    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-run-ovn\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394469    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7d87b726-552b-44e6-9c18-61b33f67d9eb-iptables-alerter-script\") pod \"iptables-alerter-r9n22\" (UID: \"7d87b726-552b-44e6-9c18-61b33f67d9eb\") " pod="openshift-network-operator/iptables-alerter-r9n22"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394500    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-cni-bin\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394525    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-run-netns\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394543    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-log-socket\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394558    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-cni-netd\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394573    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86cs4\" (UniqueName: \"kubernetes.io/projected/4b97304d-62fa-4447-81d8-525c8b6670b3-kube-api-access-86cs4\") pod \"node-ca-5j4wp\" (UID: \"4b97304d-62fa-4447-81d8-525c8b6670b3\") " pod="openshift-image-registry/node-ca-5j4wp"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394593    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-tuned\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394615    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2jr9\" (UniqueName: \"kubernetes.io/projected/559deb09-e606-40bb-ba17-b957dca7b41a-kube-api-access-b2jr9\") pod \"node-resolver-s7jjg\" (UID: \"559deb09-e606-40bb-ba17-b957dca7b41a\") " pod="openshift-dns/node-resolver-s7jjg"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394630    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b97304d-62fa-4447-81d8-525c8b6670b3-host\") pod \"node-ca-5j4wp\" (UID: \"4b97304d-62fa-4447-81d8-525c8b6670b3\") " pod="openshift-image-registry/node-ca-5j4wp"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394647    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394661    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-registration-dir\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394678    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-var-lib-openvswitch\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.395044 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394723    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b97304d-62fa-4447-81d8-525c8b6670b3-serviceca\") pod \"node-ca-5j4wp\" (UID: \"4b97304d-62fa-4447-81d8-525c8b6670b3\") " pod="openshift-image-registry/node-ca-5j4wp"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394739    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-dbus\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394773    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-sysctl-conf\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394794    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d87b726-552b-44e6-9c18-61b33f67d9eb-host-slash\") pod \"iptables-alerter-r9n22\" (UID: \"7d87b726-552b-44e6-9c18-61b33f67d9eb\") " pod="openshift-network-operator/iptables-alerter-r9n22"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394806    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-host\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394827    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwx2z\" (UniqueName: \"kubernetes.io/projected/6d26b109-0e00-4704-95a2-9cda7952facb-kube-api-access-kwx2z\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394874    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5618041c-75a6-4c3b-a584-519fad1a4b20-konnectivity-ca\") pod \"konnectivity-agent-nxrsc\" (UID: \"5618041c-75a6-4c3b-a584-519fad1a4b20\") " pod="kube-system/konnectivity-agent-nxrsc"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394914    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-kubelet-config\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394964    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.394985    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-device-dir\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.395000    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-systemd-units\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.395016    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-kubernetes\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.395037    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzdb5\" (UniqueName: \"kubernetes.io/projected/13bb334e-7e62-4d28-9211-d1b93c652016-kube-api-access-qzdb5\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.395055    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-slash\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.395070    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-run-ovn-kubernetes\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.395854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.395084    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/298d82e2-1299-49d7-a29c-1d323a2c595c-ovnkube-config\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.396921 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.395103    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-lib-modules\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.397041 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.397022    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-227.ec2.internal"]
Jul 24 03:17:26.397090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.397054    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 03:17:26.397139 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.397105    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal"
Jul 24 03:17:26.401064 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.401045    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 03:17:26.405784 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.405768    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 03:17:26.405990 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.405974    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal"]
Jul 24 03:17:26.423299 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.423244    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ngwjz"
Jul 24 03:17:26.431099 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.431080    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ngwjz"
Jul 24 03:17:26.488506 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.488485    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 03:17:26.496035 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496009    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-log-socket\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.496154 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496040    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-cni-netd\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.496154 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496059    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86cs4\" (UniqueName: \"kubernetes.io/projected/4b97304d-62fa-4447-81d8-525c8b6670b3-kube-api-access-86cs4\") pod \"node-ca-5j4wp\" (UID: \"4b97304d-62fa-4447-81d8-525c8b6670b3\") " pod="openshift-image-registry/node-ca-5j4wp"
Jul 24 03:17:26.496154 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496077    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-tuned\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.496154 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496099    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2jr9\" (UniqueName: \"kubernetes.io/projected/559deb09-e606-40bb-ba17-b957dca7b41a-kube-api-access-b2jr9\") pod \"node-resolver-s7jjg\" (UID: \"559deb09-e606-40bb-ba17-b957dca7b41a\") " pod="openshift-dns/node-resolver-s7jjg"
Jul 24 03:17:26.496154 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496124    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b97304d-62fa-4447-81d8-525c8b6670b3-host\") pod \"node-ca-5j4wp\" (UID: \"4b97304d-62fa-4447-81d8-525c8b6670b3\") " pod="openshift-image-registry/node-ca-5j4wp"
Jul 24 03:17:26.496154 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496124    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-cni-netd\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.496154 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496122    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-log-socket\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.496154 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496153    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d6301d1-c237-48f6-b5af-921a4a4540f1-system-cni-dir\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496183    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496201    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b97304d-62fa-4447-81d8-525c8b6670b3-host\") pod \"node-ca-5j4wp\" (UID: \"4b97304d-62fa-4447-81d8-525c8b6670b3\") " pod="openshift-image-registry/node-ca-5j4wp"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496209    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-registration-dir\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496257    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-registration-dir\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.496292    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496280    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-var-lib-openvswitch\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.496355    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret podName:3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d nodeName:}" failed. No retries permitted until 2026-07-24 03:17:26.99633534 +0000 UTC m=+1.959428640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret") pod "global-pull-secret-syncer-6vl7h" (UID: "3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496380    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496403    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b97304d-62fa-4447-81d8-525c8b6670b3-serviceca\") pod \"node-ca-5j4wp\" (UID: \"4b97304d-62fa-4447-81d8-525c8b6670b3\") " pod="openshift-image-registry/node-ca-5j4wp"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496410    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-var-lib-openvswitch\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496449    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496464    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496501    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm2hm\" (UniqueName: \"kubernetes.io/projected/bd424b0b-8659-412b-98d1-a0a18b15bd2d-kube-api-access-lm2hm\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496527    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-dbus\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496543    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-sysctl-conf\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496571    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d87b726-552b-44e6-9c18-61b33f67d9eb-host-slash\") pod \"iptables-alerter-r9n22\" (UID: \"7d87b726-552b-44e6-9c18-61b33f67d9eb\") " pod="openshift-network-operator/iptables-alerter-r9n22"
Jul 24 03:17:26.496611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496586    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-dbus\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496589    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-multus-cni-dir\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496618    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-var-lib-kubelet\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496623    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d87b726-552b-44e6-9c18-61b33f67d9eb-host-slash\") pod \"iptables-alerter-r9n22\" (UID: \"7d87b726-552b-44e6-9c18-61b33f67d9eb\") " pod="openshift-network-operator/iptables-alerter-r9n22"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496636    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-host\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496672    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-sysctl-conf\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496693    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-host\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496680    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwx2z\" (UniqueName: \"kubernetes.io/projected/6d26b109-0e00-4704-95a2-9cda7952facb-kube-api-access-kwx2z\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496739    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5618041c-75a6-4c3b-a584-519fad1a4b20-konnectivity-ca\") pod \"konnectivity-agent-nxrsc\" (UID: \"5618041c-75a6-4c3b-a584-519fad1a4b20\") " pod="kube-system/konnectivity-agent-nxrsc"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496766    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-run-multus-certs\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496794    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-kubelet-config\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496817    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496842    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-device-dir\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496864    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-systemd-units\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496878    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-kubelet-config\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496888    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d6301d1-c237-48f6-b5af-921a4a4540f1-cni-binary-copy\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.496909    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496926    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-systemd-units\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.497461 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496935    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b97304d-62fa-4447-81d8-525c8b6670b3-serviceca\") pod \"node-ca-5j4wp\" (UID: \"4b97304d-62fa-4447-81d8-525c8b6670b3\") " pod="openshift-image-registry/node-ca-5j4wp"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496905    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-device-dir\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.496957    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d6301d1-c237-48f6-b5af-921a4a4540f1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.496963    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs podName:6d26b109-0e00-4704-95a2-9cda7952facb nodeName:}" failed. No retries permitted until 2026-07-24 03:17:26.996949622 +0000 UTC m=+1.960042890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs") pod "network-metrics-daemon-mp65w" (UID: "6d26b109-0e00-4704-95a2-9cda7952facb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497014    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-run-netns\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497043    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-kubernetes\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497100    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-kubernetes\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497088    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzdb5\" (UniqueName: \"kubernetes.io/projected/13bb334e-7e62-4d28-9211-d1b93c652016-kube-api-access-qzdb5\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497149    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-slash\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497187    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-run-ovn-kubernetes\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497211    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/298d82e2-1299-49d7-a29c-1d323a2c595c-ovnkube-config\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497239    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-cnibin\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497240    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5618041c-75a6-4c3b-a584-519fad1a4b20-konnectivity-ca\") pod \"konnectivity-agent-nxrsc\" (UID: \"5618041c-75a6-4c3b-a584-519fad1a4b20\") " pod="kube-system/konnectivity-agent-nxrsc"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497252    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-slash\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497262    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-var-lib-cni-multus\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497309    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-run-ovn-kubernetes\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.498349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497321    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-lib-modules\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497362    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/559deb09-e606-40bb-ba17-b957dca7b41a-tmp-dir\") pod \"node-resolver-s7jjg\" (UID: \"559deb09-e606-40bb-ba17-b957dca7b41a\") " pod="openshift-dns/node-resolver-s7jjg"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497385    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-node-log\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497415    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/298d82e2-1299-49d7-a29c-1d323a2c595c-ovn-node-metrics-cert\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497485    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-os-release\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497511    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-lib-modules\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497543    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-node-log\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497556    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-run\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497602    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-sys-fs\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497645    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-kubelet\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497669    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/298d82e2-1299-49d7-a29c-1d323a2c595c-env-overrides\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497691    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-run\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497696    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0d6301d1-c237-48f6-b5af-921a4a4540f1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497732    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-sysconfig\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497746    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/559deb09-e606-40bb-ba17-b957dca7b41a-tmp-dir\") pod \"node-resolver-s7jjg\" (UID: \"559deb09-e606-40bb-ba17-b957dca7b41a\") " pod="openshift-dns/node-resolver-s7jjg"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497759    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-var-lib-kubelet\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497768    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-sys-fs\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497786    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/298d82e2-1299-49d7-a29c-1d323a2c595c-ovnkube-script-lib\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.499080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497824    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-kubelet\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497785    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/298d82e2-1299-49d7-a29c-1d323a2c595c-ovnkube-config\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497839    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0d6301d1-c237-48f6-b5af-921a4a4540f1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497835    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-var-lib-kubelet\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497874    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w97sm\" (UniqueName: \"kubernetes.io/projected/0d6301d1-c237-48f6-b5af-921a4a4540f1-kube-api-access-w97sm\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497900    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-run-k8s-cni-cncf-io\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497948    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-sysctl-d\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.497973    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/559deb09-e606-40bb-ba17-b957dca7b41a-hosts-file\") pod \"node-resolver-s7jjg\" (UID: \"559deb09-e606-40bb-ba17-b957dca7b41a\") " pod="openshift-dns/node-resolver-s7jjg"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498031    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-var-lib-cni-bin\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498056    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-kubelet-dir\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498080    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-sys\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498120    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-socket-dir\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498149    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlgbn\" (UniqueName: \"kubernetes.io/projected/7d87b726-552b-44e6-9c18-61b33f67d9eb-kube-api-access-dlgbn\") pod \"iptables-alerter-r9n22\" (UID: \"7d87b726-552b-44e6-9c18-61b33f67d9eb\") " pod="openshift-network-operator/iptables-alerter-r9n22"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498151    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-sysctl-d\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498170    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/559deb09-e606-40bb-ba17-b957dca7b41a-hosts-file\") pod \"node-resolver-s7jjg\" (UID: \"559deb09-e606-40bb-ba17-b957dca7b41a\") " pod="openshift-dns/node-resolver-s7jjg"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498176    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-etc-openvswitch\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498202    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jfsj\" (UniqueName: \"kubernetes.io/projected/298d82e2-1299-49d7-a29c-1d323a2c595c-kube-api-access-8jfsj\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.499979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498211    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/298d82e2-1299-49d7-a29c-1d323a2c595c-env-overrides\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498220    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-kubelet-dir\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498225    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd424b0b-8659-412b-98d1-a0a18b15bd2d-multus-daemon-config\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498217    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-sys\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498245    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/298d82e2-1299-49d7-a29c-1d323a2c595c-ovnkube-script-lib\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498250    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-sysconfig\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498265    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5618041c-75a6-4c3b-a584-519fad1a4b20-agent-certs\") pod \"konnectivity-agent-nxrsc\" (UID: \"5618041c-75a6-4c3b-a584-519fad1a4b20\") " pod="kube-system/konnectivity-agent-nxrsc"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498291    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d6301d1-c237-48f6-b5af-921a4a4540f1-cnibin\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498590    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-socket-dir\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498883    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-etc-openvswitch\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498931    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-multus-socket-dir-parent\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.498971    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-multus-conf-dir\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499038    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-etc-kubernetes\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499068    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-modprobe-d\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499116    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-systemd\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499146    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91d31d75-fb0f-44f5-94b5-129970db4fea-tmp\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499218    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-systemd\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.500875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499465    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc5rp\" (UniqueName: \"kubernetes.io/projected/91d31d75-fb0f-44f5-94b5-129970db4fea-kube-api-access-kc5rp\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499507    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-etc-selinux\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499534    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-run-systemd\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499563    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-run-openvswitch\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499592    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-run-ovn\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499624    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7d87b726-552b-44e6-9c18-61b33f67d9eb-iptables-alerter-script\") pod \"iptables-alerter-r9n22\" (UID: \"7d87b726-552b-44e6-9c18-61b33f67d9eb\") " pod="openshift-network-operator/iptables-alerter-r9n22"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499654    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-cni-bin\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499683    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwfk5\" (UniqueName: \"kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5\") pod \"network-check-target-5g2fw\" (UID: \"7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0\") " pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499723    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d6301d1-c237-48f6-b5af-921a4a4540f1-os-release\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499801    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-system-cni-dir\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499867    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd424b0b-8659-412b-98d1-a0a18b15bd2d-cni-binary-copy\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499923    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-hostroot\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.499969    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-run-netns\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.500078    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-run-netns\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.500140    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-host-cni-bin\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.500247    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-run-systemd\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.500292    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/298d82e2-1299-49d7-a29c-1d323a2c595c-ovn-node-metrics-cert\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.501861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.500345    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/13bb334e-7e62-4d28-9211-d1b93c652016-etc-selinux\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.502710 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.500510    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-tuned\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.502710 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.500520    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7d87b726-552b-44e6-9c18-61b33f67d9eb-iptables-alerter-script\") pod \"iptables-alerter-r9n22\" (UID: \"7d87b726-552b-44e6-9c18-61b33f67d9eb\") " pod="openshift-network-operator/iptables-alerter-r9n22"
Jul 24 03:17:26.502710 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.500585    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-run-openvswitch\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.502710 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.500632    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/298d82e2-1299-49d7-a29c-1d323a2c595c-run-ovn\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.502710 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.501509    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5618041c-75a6-4c3b-a584-519fad1a4b20-agent-certs\") pod \"konnectivity-agent-nxrsc\" (UID: \"5618041c-75a6-4c3b-a584-519fad1a4b20\") " pod="kube-system/konnectivity-agent-nxrsc"
Jul 24 03:17:26.503038 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.503011    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91d31d75-fb0f-44f5-94b5-129970db4fea-etc-modprobe-d\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.505654 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.505626    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91d31d75-fb0f-44f5-94b5-129970db4fea-tmp\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.506072 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.506046    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwx2z\" (UniqueName: \"kubernetes.io/projected/6d26b109-0e00-4704-95a2-9cda7952facb-kube-api-access-kwx2z\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:26.507208 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.507180    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc5rp\" (UniqueName: \"kubernetes.io/projected/91d31d75-fb0f-44f5-94b5-129970db4fea-kube-api-access-kc5rp\") pod \"tuned-8fxrm\" (UID: \"91d31d75-fb0f-44f5-94b5-129970db4fea\") " pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.507208 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.507184    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlgbn\" (UniqueName: \"kubernetes.io/projected/7d87b726-552b-44e6-9c18-61b33f67d9eb-kube-api-access-dlgbn\") pod \"iptables-alerter-r9n22\" (UID: \"7d87b726-552b-44e6-9c18-61b33f67d9eb\") " pod="openshift-network-operator/iptables-alerter-r9n22"
Jul 24 03:17:26.507552 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.507327    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzdb5\" (UniqueName: \"kubernetes.io/projected/13bb334e-7e62-4d28-9211-d1b93c652016-kube-api-access-qzdb5\") pod \"aws-ebs-csi-driver-node-94bk8\" (UID: \"13bb334e-7e62-4d28-9211-d1b93c652016\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.507609 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.507593    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2jr9\" (UniqueName: \"kubernetes.io/projected/559deb09-e606-40bb-ba17-b957dca7b41a-kube-api-access-b2jr9\") pod \"node-resolver-s7jjg\" (UID: \"559deb09-e606-40bb-ba17-b957dca7b41a\") " pod="openshift-dns/node-resolver-s7jjg"
Jul 24 03:17:26.507704 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.507679    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86cs4\" (UniqueName: \"kubernetes.io/projected/4b97304d-62fa-4447-81d8-525c8b6670b3-kube-api-access-86cs4\") pod \"node-ca-5j4wp\" (UID: \"4b97304d-62fa-4447-81d8-525c8b6670b3\") " pod="openshift-image-registry/node-ca-5j4wp"
Jul 24 03:17:26.508120 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.508101    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jfsj\" (UniqueName: \"kubernetes.io/projected/298d82e2-1299-49d7-a29c-1d323a2c595c-kube-api-access-8jfsj\") pod \"ovnkube-node-zgfsh\" (UID: \"298d82e2-1299-49d7-a29c-1d323a2c595c\") " pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.511793 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.511772    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s7jjg"
Jul 24 03:17:26.512196 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.512177    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 03:17:26.530413 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.530392    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5j4wp"
Jul 24 03:17:26.601029 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.600998    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-multus-cni-dir\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601029 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601030    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-var-lib-kubelet\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601050    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-run-multus-certs\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601082    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d6301d1-c237-48f6-b5af-921a4a4540f1-cni-binary-copy\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.601232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601119    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d6301d1-c237-48f6-b5af-921a4a4540f1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.601232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601124    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-var-lib-kubelet\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601131    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-run-multus-certs\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601143    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-run-netns\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601167    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-cnibin\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601170    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-run-netns\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601132    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-multus-cni-dir\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601184    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-var-lib-cni-multus\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601217    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-os-release\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601220    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-var-lib-cni-multus\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601221    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-cnibin\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601251    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0d6301d1-c237-48f6-b5af-921a4a4540f1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601274    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d6301d1-c237-48f6-b5af-921a4a4540f1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601283    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0d6301d1-c237-48f6-b5af-921a4a4540f1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601296    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-os-release\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601336    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w97sm\" (UniqueName: \"kubernetes.io/projected/0d6301d1-c237-48f6-b5af-921a4a4540f1-kube-api-access-w97sm\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601361    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-run-k8s-cni-cncf-io\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601379    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-var-lib-cni-bin\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601408    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd424b0b-8659-412b-98d1-a0a18b15bd2d-multus-daemon-config\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601450    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-run-k8s-cni-cncf-io\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601457    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-host-var-lib-cni-bin\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601476    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d6301d1-c237-48f6-b5af-921a4a4540f1-cnibin\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601492    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-multus-socket-dir-parent\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601509    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-multus-conf-dir\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601526    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-etc-kubernetes\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601530    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d6301d1-c237-48f6-b5af-921a4a4540f1-cnibin\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601551    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwfk5\" (UniqueName: \"kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5\") pod \"network-check-target-5g2fw\" (UID: \"7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0\") " pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:26.601683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601564    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-multus-socket-dir-parent\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601568    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d6301d1-c237-48f6-b5af-921a4a4540f1-os-release\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601578    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-multus-conf-dir\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601608    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-system-cni-dir\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601616    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-etc-kubernetes\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601633    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd424b0b-8659-412b-98d1-a0a18b15bd2d-cni-binary-copy\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601656    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-hostroot\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601668    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d6301d1-c237-48f6-b5af-921a4a4540f1-os-release\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601703    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-hostroot\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601754    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd424b0b-8659-412b-98d1-a0a18b15bd2d-system-cni-dir\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.601963    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d6301d1-c237-48f6-b5af-921a4a4540f1-system-cni-dir\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.602029    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm2hm\" (UniqueName: \"kubernetes.io/projected/bd424b0b-8659-412b-98d1-a0a18b15bd2d-kube-api-access-lm2hm\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.602058    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d6301d1-c237-48f6-b5af-921a4a4540f1-system-cni-dir\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.602201    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d6301d1-c237-48f6-b5af-921a4a4540f1-cni-binary-copy\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.602309    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0d6301d1-c237-48f6-b5af-921a4a4540f1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.602347    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0d6301d1-c237-48f6-b5af-921a4a4540f1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.602363    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd424b0b-8659-412b-98d1-a0a18b15bd2d-cni-binary-copy\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.602417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.602400    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd424b0b-8659-412b-98d1-a0a18b15bd2d-multus-daemon-config\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.609173 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.609148    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:26.609173 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.609172    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:26.609173 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.609182    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qwfk5 for pod openshift-network-diagnostics/network-check-target-5g2fw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:26.609497 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:26.609238    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5 podName:7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:27.109223813 +0000 UTC m=+2.072317096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qwfk5" (UniqueName: "kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5") pod "network-check-target-5g2fw" (UID: "7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:26.610494 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.610471    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm2hm\" (UniqueName: \"kubernetes.io/projected/bd424b0b-8659-412b-98d1-a0a18b15bd2d-kube-api-access-lm2hm\") pod \"multus-gx5vw\" (UID: \"bd424b0b-8659-412b-98d1-a0a18b15bd2d\") " pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.611772 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.611753    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w97sm\" (UniqueName: \"kubernetes.io/projected/0d6301d1-c237-48f6-b5af-921a4a4540f1-kube-api-access-w97sm\") pod \"multus-additional-cni-plugins-4wrj9\" (UID: \"0d6301d1-c237-48f6-b5af-921a4a4540f1\") " pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.653634 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:26.653598    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbceac0aa275a63572a2fe0294b42f48b.slice/crio-c796b93375143ba231993a286f91dfb773b05053e20e4ab0789818b7acfc85ff WatchSource:0}: Error finding container c796b93375143ba231993a286f91dfb773b05053e20e4ab0789818b7acfc85ff: Status 404 returned error can't find the container with id c796b93375143ba231993a286f91dfb773b05053e20e4ab0789818b7acfc85ff
Jul 24 03:17:26.654098 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:26.654073    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9eba28ae5571b83e98bdb2f8bef7f3.slice/crio-69032696a5e7bd560fab0114506bedf03f7f4db4884ff4be2c5874a4a22d8e4e WatchSource:0}: Error finding container 69032696a5e7bd560fab0114506bedf03f7f4db4884ff4be2c5874a4a22d8e4e: Status 404 returned error can't find the container with id 69032696a5e7bd560fab0114506bedf03f7f4db4884ff4be2c5874a4a22d8e4e
Jul 24 03:17:26.657931 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.657912    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 03:17:26.702714 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.702615    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r9n22"
Jul 24 03:17:26.712501 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:26.712470    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d87b726_552b_44e6_9c18_61b33f67d9eb.slice/crio-b395442485e6598f77b98f2330fadcfaa7c4334a5fabb2f8dea69701f0ee83cb WatchSource:0}: Error finding container b395442485e6598f77b98f2330fadcfaa7c4334a5fabb2f8dea69701f0ee83cb: Status 404 returned error can't find the container with id b395442485e6598f77b98f2330fadcfaa7c4334a5fabb2f8dea69701f0ee83cb
Jul 24 03:17:26.725091 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.725061    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:26.734510 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:26.734481    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod298d82e2_1299_49d7_a29c_1d323a2c595c.slice/crio-b1c37551206fb565de016906efab87988aaf7b3848bbd6e1fb749657045f3bd1 WatchSource:0}: Error finding container b1c37551206fb565de016906efab87988aaf7b3848bbd6e1fb749657045f3bd1: Status 404 returned error can't find the container with id b1c37551206fb565de016906efab87988aaf7b3848bbd6e1fb749657045f3bd1
Jul 24 03:17:26.746748 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.746716    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nxrsc"
Jul 24 03:17:26.755714 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:26.755688    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5618041c_75a6_4c3b_a584_519fad1a4b20.slice/crio-06d9e06f66c1cc43a355e1a8e3243f33a91a1a67eb0f056bb7f80b18890a18ed WatchSource:0}: Error finding container 06d9e06f66c1cc43a355e1a8e3243f33a91a1a67eb0f056bb7f80b18890a18ed: Status 404 returned error can't find the container with id 06d9e06f66c1cc43a355e1a8e3243f33a91a1a67eb0f056bb7f80b18890a18ed
Jul 24 03:17:26.765543 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.765486    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8fxrm"
Jul 24 03:17:26.775665 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:26.775639    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91d31d75_fb0f_44f5_94b5_129970db4fea.slice/crio-3f5a511b21c4ab67292f9a0a4c5be7a4b42e5b95c83662766bdcf5eeea71cb3f WatchSource:0}: Error finding container 3f5a511b21c4ab67292f9a0a4c5be7a4b42e5b95c83662766bdcf5eeea71cb3f: Status 404 returned error can't find the container with id 3f5a511b21c4ab67292f9a0a4c5be7a4b42e5b95c83662766bdcf5eeea71cb3f
Jul 24 03:17:26.781492 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.781468    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8"
Jul 24 03:17:26.790779 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:26.790743    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13bb334e_7e62_4d28_9211_d1b93c652016.slice/crio-abf8f7eb980493ef3b7a9c367f20ed697c9f4dcfbdd0acd0f5d8c54051c47b99 WatchSource:0}: Error finding container abf8f7eb980493ef3b7a9c367f20ed697c9f4dcfbdd0acd0f5d8c54051c47b99: Status 404 returned error can't find the container with id abf8f7eb980493ef3b7a9c367f20ed697c9f4dcfbdd0acd0f5d8c54051c47b99
Jul 24 03:17:26.837019 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.836973    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4wrj9"
Jul 24 03:17:26.846197 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:26.846167    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d6301d1_c237_48f6_b5af_921a4a4540f1.slice/crio-3eae8db614dd239c7935ebf634f5d911ae98a63fedb1bc39577d2225bf1eab75 WatchSource:0}: Error finding container 3eae8db614dd239c7935ebf634f5d911ae98a63fedb1bc39577d2225bf1eab75: Status 404 returned error can't find the container with id 3eae8db614dd239c7935ebf634f5d911ae98a63fedb1bc39577d2225bf1eab75
Jul 24 03:17:26.868173 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:26.868139    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gx5vw"
Jul 24 03:17:26.877619 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:26.877592    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd424b0b_8659_412b_98d1_a0a18b15bd2d.slice/crio-5c519f798f81dfbff7d03e1ae24b9a6d6417d451c832274f5eab47df6bacf941 WatchSource:0}: Error finding container 5c519f798f81dfbff7d03e1ae24b9a6d6417d451c832274f5eab47df6bacf941: Status 404 returned error can't find the container with id 5c519f798f81dfbff7d03e1ae24b9a6d6417d451c832274f5eab47df6bacf941
Jul 24 03:17:26.927183 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:26.927151    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b97304d_62fa_4447_81d8_525c8b6670b3.slice/crio-9e5c6f868add720fd92dfdcc476d69f1e0c4224d3f3c415283eaa78743e03348 WatchSource:0}: Error finding container 9e5c6f868add720fd92dfdcc476d69f1e0c4224d3f3c415283eaa78743e03348: Status 404 returned error can't find the container with id 9e5c6f868add720fd92dfdcc476d69f1e0c4224d3f3c415283eaa78743e03348
Jul 24 03:17:27.004690 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.004593    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:27.004690 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.004638    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:27.004893 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:27.004739    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:27.004893 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:27.004746    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:27.004893 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:27.004796    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs podName:6d26b109-0e00-4704-95a2-9cda7952facb nodeName:}" failed. No retries permitted until 2026-07-24 03:17:28.004780431 +0000 UTC m=+2.967873698 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs") pod "network-metrics-daemon-mp65w" (UID: "6d26b109-0e00-4704-95a2-9cda7952facb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:27.004893 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:27.004824    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret podName:3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d nodeName:}" failed. No retries permitted until 2026-07-24 03:17:28.00480898 +0000 UTC m=+2.967902246 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret") pod "global-pull-secret-syncer-6vl7h" (UID: "3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:27.122085 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:17:27.122050    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod559deb09_e606_40bb_ba17_b957dca7b41a.slice/crio-5d8914f855a11ebaa92927995f91126027abec73cc12ff06972637e407410330 WatchSource:0}: Error finding container 5d8914f855a11ebaa92927995f91126027abec73cc12ff06972637e407410330: Status 404 returned error can't find the container with id 5d8914f855a11ebaa92927995f91126027abec73cc12ff06972637e407410330
Jul 24 03:17:27.206422 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.206380    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwfk5\" (UniqueName: \"kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5\") pod \"network-check-target-5g2fw\" (UID: \"7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0\") " pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:27.206639 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:27.206571    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:27.206639 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:27.206604    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:27.206639 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:27.206619    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qwfk5 for pod openshift-network-diagnostics/network-check-target-5g2fw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:27.206787 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:27.206694    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5 podName:7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:28.206672772 +0000 UTC m=+3.169766041 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwfk5" (UniqueName: "kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5") pod "network-check-target-5g2fw" (UID: "7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:27.275727 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.275392    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 03:17:27.433594 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.433420    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 03:12:27 +0000 UTC" deadline="2028-01-31 13:33:10.344117958 +0000 UTC"
Jul 24 03:17:27.433594 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.433534    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13354h15m42.910589584s"
Jul 24 03:17:27.570434 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.570349    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:27.570626 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:27.570530    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:27.615997 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.615581    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8fxrm" event={"ID":"91d31d75-fb0f-44f5-94b5-129970db4fea","Type":"ContainerStarted","Data":"3f5a511b21c4ab67292f9a0a4c5be7a4b42e5b95c83662766bdcf5eeea71cb3f"}
Jul 24 03:17:27.630177 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.630135    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nxrsc" event={"ID":"5618041c-75a6-4c3b-a584-519fad1a4b20","Type":"ContainerStarted","Data":"06d9e06f66c1cc43a355e1a8e3243f33a91a1a67eb0f056bb7f80b18890a18ed"}
Jul 24 03:17:27.652012 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.651967    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r9n22" event={"ID":"7d87b726-552b-44e6-9c18-61b33f67d9eb","Type":"ContainerStarted","Data":"b395442485e6598f77b98f2330fadcfaa7c4334a5fabb2f8dea69701f0ee83cb"}
Jul 24 03:17:27.658609 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.658566    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s7jjg" event={"ID":"559deb09-e606-40bb-ba17-b957dca7b41a","Type":"ContainerStarted","Data":"5d8914f855a11ebaa92927995f91126027abec73cc12ff06972637e407410330"}
Jul 24 03:17:27.671405 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.671362    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gx5vw" event={"ID":"bd424b0b-8659-412b-98d1-a0a18b15bd2d","Type":"ContainerStarted","Data":"5c519f798f81dfbff7d03e1ae24b9a6d6417d451c832274f5eab47df6bacf941"}
Jul 24 03:17:27.683459 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.683406    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wrj9" event={"ID":"0d6301d1-c237-48f6-b5af-921a4a4540f1","Type":"ContainerStarted","Data":"3eae8db614dd239c7935ebf634f5d911ae98a63fedb1bc39577d2225bf1eab75"}
Jul 24 03:17:27.722718 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.722676    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8" event={"ID":"13bb334e-7e62-4d28-9211-d1b93c652016","Type":"ContainerStarted","Data":"abf8f7eb980493ef3b7a9c367f20ed697c9f4dcfbdd0acd0f5d8c54051c47b99"}
Jul 24 03:17:27.728238 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.728210    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 03:17:27.753062 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.753022    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh" event={"ID":"298d82e2-1299-49d7-a29c-1d323a2c595c","Type":"ContainerStarted","Data":"b1c37551206fb565de016906efab87988aaf7b3848bbd6e1fb749657045f3bd1"}
Jul 24 03:17:27.760050 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.760009    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal" event={"ID":"bceac0aa275a63572a2fe0294b42f48b","Type":"ContainerStarted","Data":"c796b93375143ba231993a286f91dfb773b05053e20e4ab0789818b7acfc85ff"}
Jul 24 03:17:27.773645 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.773597    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-227.ec2.internal" event={"ID":"cb9eba28ae5571b83e98bdb2f8bef7f3","Type":"ContainerStarted","Data":"69032696a5e7bd560fab0114506bedf03f7f4db4884ff4be2c5874a4a22d8e4e"}
Jul 24 03:17:27.783429 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:27.783388    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5j4wp" event={"ID":"4b97304d-62fa-4447-81d8-525c8b6670b3","Type":"ContainerStarted","Data":"9e5c6f868add720fd92dfdcc476d69f1e0c4224d3f3c415283eaa78743e03348"}
Jul 24 03:17:28.012982 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:28.012885    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:28.013161 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:28.013004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:28.013161 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:28.013139    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:28.013266 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:28.013205    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret podName:3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d nodeName:}" failed. No retries permitted until 2026-07-24 03:17:30.013185751 +0000 UTC m=+4.976279024 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret") pod "global-pull-secret-syncer-6vl7h" (UID: "3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:28.014153 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:28.013695    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:28.014153 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:28.013758    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs podName:6d26b109-0e00-4704-95a2-9cda7952facb nodeName:}" failed. No retries permitted until 2026-07-24 03:17:30.013740466 +0000 UTC m=+4.976833735 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs") pod "network-metrics-daemon-mp65w" (UID: "6d26b109-0e00-4704-95a2-9cda7952facb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:28.215119 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:28.215084    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwfk5\" (UniqueName: \"kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5\") pod \"network-check-target-5g2fw\" (UID: \"7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0\") " pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:28.215349 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:28.215235    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:28.215349 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:28.215253    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:28.215349 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:28.215267    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qwfk5 for pod openshift-network-diagnostics/network-check-target-5g2fw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:28.215349 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:28.215332    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5 podName:7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:30.215311969 +0000 UTC m=+5.178405244 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwfk5" (UniqueName: "kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5") pod "network-check-target-5g2fw" (UID: "7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:28.390544 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:28.390298    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 03:17:28.433737 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:28.433679    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 03:12:27 +0000 UTC" deadline="2028-01-26 19:24:43.627108807 +0000 UTC"
Jul 24 03:17:28.433737 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:28.433727    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13240h7m15.193387326s"
Jul 24 03:17:28.567979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:28.567188    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:28.567979 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:28.567348    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:28.567979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:28.567805    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:28.567979 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:28.567914    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:29.570056 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:29.569751    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:29.570560 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:29.570195    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:30.031765 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:30.031666    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:30.031765 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:30.031733    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:30.032007 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:30.031876    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:30.032007 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:30.031944    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs podName:6d26b109-0e00-4704-95a2-9cda7952facb nodeName:}" failed. No retries permitted until 2026-07-24 03:17:34.031924072 +0000 UTC m=+8.995017345 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs") pod "network-metrics-daemon-mp65w" (UID: "6d26b109-0e00-4704-95a2-9cda7952facb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:30.032143 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:30.032009    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:30.032143 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:30.032041    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret podName:3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d nodeName:}" failed. No retries permitted until 2026-07-24 03:17:34.032030559 +0000 UTC m=+8.995123828 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret") pod "global-pull-secret-syncer-6vl7h" (UID: "3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:30.233392 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:30.233344    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwfk5\" (UniqueName: \"kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5\") pod \"network-check-target-5g2fw\" (UID: \"7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0\") " pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:30.233640 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:30.233619    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:30.233703 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:30.233646    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:30.233703 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:30.233661    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qwfk5 for pod openshift-network-diagnostics/network-check-target-5g2fw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:30.233785 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:30.233729    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5 podName:7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:34.233706942 +0000 UTC m=+9.196800231 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwfk5" (UniqueName: "kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5") pod "network-check-target-5g2fw" (UID: "7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:30.567525 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:30.567473    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:30.567525 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:30.567498    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:30.567793 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:30.567620    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:30.567793 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:30.567757    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:31.567683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:31.567020    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:31.567683 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:31.567178    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:32.567174 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:32.567057    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:32.567174 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:32.567117    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:32.567603 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:32.567224    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:32.567746 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:32.567704    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:33.568654 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:33.568622    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:33.569130 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:33.568770    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:34.068345 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:34.068252    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:34.068345 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:34.068316    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:34.068613 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:34.068458    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:34.068613 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:34.068547    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs podName:6d26b109-0e00-4704-95a2-9cda7952facb nodeName:}" failed. No retries permitted until 2026-07-24 03:17:42.068526516 +0000 UTC m=+17.031619799 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs") pod "network-metrics-daemon-mp65w" (UID: "6d26b109-0e00-4704-95a2-9cda7952facb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:34.069010 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:34.068989    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:34.069105 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:34.069053    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret podName:3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d nodeName:}" failed. No retries permitted until 2026-07-24 03:17:42.069036927 +0000 UTC m=+17.032130195 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret") pod "global-pull-secret-syncer-6vl7h" (UID: "3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:34.270304 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:34.270258    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwfk5\" (UniqueName: \"kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5\") pod \"network-check-target-5g2fw\" (UID: \"7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0\") " pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:34.272066 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:34.272033    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:34.272229 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:34.272073    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:34.272229 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:34.272095    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qwfk5 for pod openshift-network-diagnostics/network-check-target-5g2fw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:34.272229 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:34.272169    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5 podName:7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:42.272135696 +0000 UTC m=+17.235228977 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwfk5" (UniqueName: "kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5") pod "network-check-target-5g2fw" (UID: "7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:34.567515 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:34.566703    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:34.567515 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:34.566847    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:34.567515 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:34.567274    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:34.567515 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:34.567390    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:35.569116 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:35.569035    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:35.569691 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:35.569186    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:36.566745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:36.566700    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:36.566745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:36.566752    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:36.566991 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:36.566860    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:36.566991 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:36.566927    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:37.567199 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:37.567161    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:37.567634 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:37.567312    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:38.567612 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:38.567576    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:38.568064 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:38.567710    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:38.568064 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:38.567769    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:38.568064 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:38.567866    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:39.570070 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:39.569995    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:39.570507 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:39.570115    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:40.567497 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:40.567452    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:40.567497 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:40.567498    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:40.567725 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:40.567586    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:40.567725 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:40.567692    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:41.569700 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:41.569668    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:41.570082 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:41.569811    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:42.132993 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:42.132950    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:42.133198 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:42.133012    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:42.133198 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:42.133117    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:42.133198 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:42.133149    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:42.133198 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:42.133195    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret podName:3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d nodeName:}" failed. No retries permitted until 2026-07-24 03:17:58.133173992 +0000 UTC m=+33.096267256 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret") pod "global-pull-secret-syncer-6vl7h" (UID: "3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:42.133421 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:42.133218    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs podName:6d26b109-0e00-4704-95a2-9cda7952facb nodeName:}" failed. No retries permitted until 2026-07-24 03:17:58.133206019 +0000 UTC m=+33.096299288 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs") pod "network-metrics-daemon-mp65w" (UID: "6d26b109-0e00-4704-95a2-9cda7952facb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:42.334767 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:42.334733    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwfk5\" (UniqueName: \"kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5\") pod \"network-check-target-5g2fw\" (UID: \"7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0\") " pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:42.334939 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:42.334875    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:42.334939 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:42.334897    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:42.334939 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:42.334910    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qwfk5 for pod openshift-network-diagnostics/network-check-target-5g2fw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:42.335048 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:42.334967    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5 podName:7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0 nodeName:}" failed. No retries permitted until 2026-07-24 03:17:58.334954495 +0000 UTC m=+33.298047760 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwfk5" (UniqueName: "kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5") pod "network-check-target-5g2fw" (UID: "7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:42.567082 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:42.567001    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:42.567082 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:42.567051    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:42.567277 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:42.567140    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:42.567277 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:42.567257    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:43.570168 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:43.570093    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:43.570581 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:43.570225    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:44.567601 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:44.567556    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:44.567811 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:44.567566    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:44.567811 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:44.567714    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:44.567811 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:44.567798    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:45.567912 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:45.567872    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:45.568392 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:45.567995    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:46.567311 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:46.567266    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:46.567531 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:46.567447    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:46.567531 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:46.567493    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:46.567646 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:46.567621    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:47.567196 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:47.567168    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:47.567590 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:47.567286    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:47.863502 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:47.862295    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gx5vw" event={"ID":"bd424b0b-8659-412b-98d1-a0a18b15bd2d","Type":"ContainerStarted","Data":"1ae300376197d376675f12f5d4b01b766fdc3fc0ee79560b8be9aea5e0a5d7be"}
Jul 24 03:17:47.869670 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:47.869395    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh" event={"ID":"298d82e2-1299-49d7-a29c-1d323a2c595c","Type":"ContainerStarted","Data":"179b5f81479ee5c959a0f5f174677bfa97a8b8ba43161963677a3a7e16531288"}
Jul 24 03:17:47.870027 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:47.870003    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh" event={"ID":"298d82e2-1299-49d7-a29c-1d323a2c595c","Type":"ContainerStarted","Data":"c4edde78033ffd8e55cd3f9db4fe8c963cd3846b6dd09597d0b284808e35ff21"}
Jul 24 03:17:47.874301 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:47.873538    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-227.ec2.internal" event={"ID":"cb9eba28ae5571b83e98bdb2f8bef7f3","Type":"ContainerStarted","Data":"d05d946583396feb9630a125d91236e1f3c100533604fc01dfafdbfa10006d22"}
Jul 24 03:17:47.881445 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:47.881262    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gx5vw" podStartSLOduration=1.085256116 podStartE2EDuration="21.881241016s" totalImagesPullingTime="20.7959849s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:47.88056601 +0000 UTC m=+22.843659309" watchObservedRunningTime="2026-07-24 03:17:47.881241016 +0000 UTC m=+22.844334304"
Jul 24 03:17:47.884036 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:47.882232    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8fxrm" event={"ID":"91d31d75-fb0f-44f5-94b5-129970db4fea","Type":"ContainerStarted","Data":"b51422260738073668897ab952f71a0cda044652f4aab77402c4ec335cb18c56"}
Jul 24 03:17:47.893193 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:47.893150    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-227.ec2.internal" podStartSLOduration=21.89313503 podStartE2EDuration="21.89313503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:47.892506675 +0000 UTC m=+22.855599963" watchObservedRunningTime="2026-07-24 03:17:47.89313503 +0000 UTC m=+22.856228316"
Jul 24 03:17:47.909345 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:47.908242    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8fxrm" podStartSLOduration=1.202846369 podStartE2EDuration="21.908220895s" totalImagesPullingTime="20.705374526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:47.908126872 +0000 UTC m=+22.871220173" watchObservedRunningTime="2026-07-24 03:17:47.908220895 +0000 UTC m=+22.871314184"
Jul 24 03:17:48.567076 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.566887    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:48.567247 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.566887    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:48.567247 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:48.567156    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:48.567247 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:48.567233    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:48.886537 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.886500    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s7jjg" event={"ID":"559deb09-e606-40bb-ba17-b957dca7b41a","Type":"ContainerStarted","Data":"316c5ccbe336cdc8db5ddcdaf8c1bbd0e470a5973ae9fc976ce2091f86b6f4ff"}
Jul 24 03:17:48.888140 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.888112    2574 generic.go:356] "Generic (PLEG): container finished" podID="0d6301d1-c237-48f6-b5af-921a4a4540f1" containerID="15a42b60b56daee3774e47c44f08eb5e3170bf07735c465f16adee48076cef47" exitCode=0
Jul 24 03:17:48.888301 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.888196    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wrj9" event={"ID":"0d6301d1-c237-48f6-b5af-921a4a4540f1","Type":"ContainerDied","Data":"15a42b60b56daee3774e47c44f08eb5e3170bf07735c465f16adee48076cef47"}
Jul 24 03:17:48.892949 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.891749    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8" event={"ID":"13bb334e-7e62-4d28-9211-d1b93c652016","Type":"ContainerStarted","Data":"53e7b7edb81c4697520d5ab70aab746e3488b50ecf4fa382bc71f8049095a437"}
Jul 24 03:17:48.895970 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.895942    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh" event={"ID":"298d82e2-1299-49d7-a29c-1d323a2c595c","Type":"ContainerStarted","Data":"a91d85e68bfeea0e0461f5a86eed9eff6d24e8ba66630360a51007d35c1c9192"}
Jul 24 03:17:48.896081 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.896013    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh" event={"ID":"298d82e2-1299-49d7-a29c-1d323a2c595c","Type":"ContainerStarted","Data":"dc96c4b1c48721ad486e21c39cc9da00c51580c9e0793407290e4865386679ae"}
Jul 24 03:17:48.896081 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.896061    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh" event={"ID":"298d82e2-1299-49d7-a29c-1d323a2c595c","Type":"ContainerStarted","Data":"977798e9c0517fef4732a3f555ee29a7ee891cdafb105eac9a77ae652b4872cf"}
Jul 24 03:17:48.896172 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.896106    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh" event={"ID":"298d82e2-1299-49d7-a29c-1d323a2c595c","Type":"ContainerStarted","Data":"47ab90da0d5359954df29d2c8c32413a5e50f0141be2b141ea32a3bcdaf2d3e5"}
Jul 24 03:17:48.897576 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.897551    2574 generic.go:356] "Generic (PLEG): container finished" podID="bceac0aa275a63572a2fe0294b42f48b" containerID="440e664c9c537a6c2266bf18920414ee6520184dc61e0ad9acc1fd3eddae5059" exitCode=0
Jul 24 03:17:48.897680 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.897635    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal" event={"ID":"bceac0aa275a63572a2fe0294b42f48b","Type":"ContainerDied","Data":"440e664c9c537a6c2266bf18920414ee6520184dc61e0ad9acc1fd3eddae5059"}
Jul 24 03:17:48.899226 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.899192    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5j4wp" event={"ID":"4b97304d-62fa-4447-81d8-525c8b6670b3","Type":"ContainerStarted","Data":"dfaffa5c59b898fce130ff0f12333064e74bb19ac96a93f6f775a5b4d82bf31b"}
Jul 24 03:17:48.900943 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.900917    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nxrsc" event={"ID":"5618041c-75a6-4c3b-a584-519fad1a4b20","Type":"ContainerStarted","Data":"82dd68da0cfc712e50bebce4cfbacc5615a54f52f9bef05f70d91794b8ee032c"}
Jul 24 03:17:48.906795 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.906746    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s7jjg" podStartSLOduration=2.840136684 podStartE2EDuration="22.906729311s" totalImagesPullingTime="20.066592627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:48.906326993 +0000 UTC m=+23.869420280" watchObservedRunningTime="2026-07-24 03:17:48.906729311 +0000 UTC m=+23.869822598"
Jul 24 03:17:48.944356 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.944296    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nxrsc" podStartSLOduration=2.245665269 podStartE2EDuration="22.944276013s" totalImagesPullingTime="20.698610744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:48.943574003 +0000 UTC m=+23.906667290" watchObservedRunningTime="2026-07-24 03:17:48.944276013 +0000 UTC m=+23.907369301"
Jul 24 03:17:48.957171 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:48.957122    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5j4wp" podStartSLOduration=2.407836626 podStartE2EDuration="22.957104782s" totalImagesPullingTime="20.549268156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:48.95688767 +0000 UTC m=+23.919980957" watchObservedRunningTime="2026-07-24 03:17:48.957104782 +0000 UTC m=+23.920198069"
Jul 24 03:17:49.032480 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.032443    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nxrsc_5618041c-75a6-4c3b-a584-519fad1a4b20/konnectivity-agent/0.log" limit=1024
Jul 24 03:17:49.117633 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.117596    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nxrsc_5618041c-75a6-4c3b-a584-519fad1a4b20/konnectivity-agent/0.log" limit=1024
Jul 24 03:17:49.399878 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.399857    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 03:17:49.457468 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.457299    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T03:17:49.399874216Z","UUID":"977c114e-c128-4e1f-b1a9-aca5b1533e61","Handler":null,"Name":"","Endpoint":""}
Jul 24 03:17:49.458982 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.458962    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 03:17:49.459123 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.459004    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 03:17:49.567099 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.567062    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:49.567275 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:49.567212    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:49.718395 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.718303    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nxrsc"
Jul 24 03:17:49.718940 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.718919    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nxrsc"
Jul 24 03:17:49.906006 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.905974    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8" event={"ID":"13bb334e-7e62-4d28-9211-d1b93c652016","Type":"ContainerStarted","Data":"ea8218a09d854fbaab3516182f693c91befe86db6916d254e72487c2e92a8f1d"}
Jul 24 03:17:49.907985 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.907955    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal" event={"ID":"bceac0aa275a63572a2fe0294b42f48b","Type":"ContainerStarted","Data":"c3ebeb6c7b68542424e483dd76dff9d35ffd728a2ef40ea1ec2084fb479a76cb"}
Jul 24 03:17:49.909561 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.909535    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r9n22" event={"ID":"7d87b726-552b-44e6-9c18-61b33f67d9eb","Type":"ContainerStarted","Data":"4e9393f09bd676366e70b6a274126d8e02cf0794807062b62e27bca92c1f18ba"}
Jul 24 03:17:49.921886 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.921846    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-227.ec2.internal" podStartSLOduration=23.921831483 podStartE2EDuration="23.921831483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:49.921453689 +0000 UTC m=+24.884546977" watchObservedRunningTime="2026-07-24 03:17:49.921831483 +0000 UTC m=+24.884924771"
Jul 24 03:17:49.934158 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:49.934112    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r9n22" podStartSLOduration=3.169857991 podStartE2EDuration="23.934098504s" totalImagesPullingTime="20.764240513s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:49.933614024 +0000 UTC m=+24.896707310" watchObservedRunningTime="2026-07-24 03:17:49.934098504 +0000 UTC m=+24.897191823"
Jul 24 03:17:50.090595 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:50.090500    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nxrsc_5618041c-75a6-4c3b-a584-519fad1a4b20/konnectivity-agent/0.log" limit=1024
Jul 24 03:17:50.140768 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:50.140733    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nxrsc_5618041c-75a6-4c3b-a584-519fad1a4b20/konnectivity-agent/0.log" limit=1024
Jul 24 03:17:50.566731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:50.566703    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:50.566890 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:50.566837    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:50.566890 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:50.566838    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:50.567155 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:50.566954    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:50.916050 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:50.916023    2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 03:17:50.916050 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:50.916023    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh" event={"ID":"298d82e2-1299-49d7-a29c-1d323a2c595c","Type":"ContainerStarted","Data":"d2d7eccdb6138f857d0f09a4dd325de4170b2569e6f3f00df2e629721da62b54"}
Jul 24 03:17:51.566960 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:51.566926    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:51.567129 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:51.567032    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:51.921617 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:51.921524    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8" event={"ID":"13bb334e-7e62-4d28-9211-d1b93c652016","Type":"ContainerStarted","Data":"1005c66073a781ac1bfb34299742bcbad57c819b950fc57abfe99344dd45164e"}
Jul 24 03:17:51.938481 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:51.938402    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-94bk8" podStartSLOduration=2.58961884 podStartE2EDuration="25.938380848s" totalImagesPullingTime="23.348762008s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:51.937237422 +0000 UTC m=+26.900330722" watchObservedRunningTime="2026-07-24 03:17:51.938380848 +0000 UTC m=+26.901474133"
Jul 24 03:17:52.567538 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:52.567491    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:52.567538 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:52.567519    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:52.567835 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:52.567641    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:52.567835 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:52.567726    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:52.928821 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:52.928582    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh" event={"ID":"298d82e2-1299-49d7-a29c-1d323a2c595c","Type":"ContainerStarted","Data":"7a9624b1a8ba7dfcdc215f059881c295c22460bf2c4b3a55aeeddc5b25b44b25"}
Jul 24 03:17:52.957828 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:52.957772    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh" podStartSLOduration=6.174416043 podStartE2EDuration="26.95775044s" totalImagesPullingTime="20.783334397s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:17:52.956975082 +0000 UTC m=+27.920068367" watchObservedRunningTime="2026-07-24 03:17:52.95775044 +0000 UTC m=+27.920843729"
Jul 24 03:17:53.289773 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:53.289680    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nxrsc_5618041c-75a6-4c3b-a584-519fad1a4b20/konnectivity-agent/0.log" limit=1024
Jul 24 03:17:53.390669 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:53.390615    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nxrsc_5618041c-75a6-4c3b-a584-519fad1a4b20/konnectivity-agent/0.log" limit=1024
Jul 24 03:17:53.567026 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:53.566946    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:53.567174 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:53.567094    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:53.931556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:53.931465    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:53.931556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:53.931554    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:53.932122 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:53.931599    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:53.949055 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:53.949025    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:53.949496 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:53.949465    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:17:54.165924 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:54.165888    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nxrsc"
Jul 24 03:17:54.166104 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:54.166036    2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 03:17:54.175120 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:54.174820    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nxrsc"
Jul 24 03:17:54.256730 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:54.256635    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nxrsc_5618041c-75a6-4c3b-a584-519fad1a4b20/konnectivity-agent/0.log" limit=1024
Jul 24 03:17:54.567644 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:54.567552    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:54.567644 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:54.567581    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:54.567837 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:54.567711    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:54.567890 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:54.567854    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:54.935907 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:54.935709    2574 generic.go:356] "Generic (PLEG): container finished" podID="0d6301d1-c237-48f6-b5af-921a4a4540f1" containerID="f34ba8ef729bd88fe717ecbbde632942c4b1db73a8a51b86a9098a26a1ff3615" exitCode=0
Jul 24 03:17:54.936565 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:54.935789    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wrj9" event={"ID":"0d6301d1-c237-48f6-b5af-921a4a4540f1","Type":"ContainerDied","Data":"f34ba8ef729bd88fe717ecbbde632942c4b1db73a8a51b86a9098a26a1ff3615"}
Jul 24 03:17:55.081710 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:55.081674    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6vl7h"]
Jul 24 03:17:55.081881 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:55.081851    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:55.081990 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:55.081968    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:55.085638 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:55.085557    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mp65w"]
Jul 24 03:17:55.085790 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:55.085696    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:55.085844 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:55.085803    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:55.086184 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:55.086162    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5g2fw"]
Jul 24 03:17:55.086280 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:55.086269    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:55.086404 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:55.086382    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:56.567193 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:56.567159    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:56.567607 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:56.567258    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:56.567607 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:56.567289    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:56.567607 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:56.567290    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:56.567607 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:56.567383    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:56.567607 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:56.567466    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:17:56.942567 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:56.942537    2574 generic.go:356] "Generic (PLEG): container finished" podID="0d6301d1-c237-48f6-b5af-921a4a4540f1" containerID="5968605f40f49a4d431475869e81e3b1befaff02c3f42b51a363dfb3cf802463" exitCode=0
Jul 24 03:17:56.942711 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:56.942597    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wrj9" event={"ID":"0d6301d1-c237-48f6-b5af-921a4a4540f1","Type":"ContainerDied","Data":"5968605f40f49a4d431475869e81e3b1befaff02c3f42b51a363dfb3cf802463"}
Jul 24 03:17:57.947178 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:57.947089    2574 generic.go:356] "Generic (PLEG): container finished" podID="0d6301d1-c237-48f6-b5af-921a4a4540f1" containerID="acaddf93f8b9eec191d5161e147a6c2e0b6c4dbad5c5c90bd5182e20774f9f63" exitCode=0
Jul 24 03:17:57.947178 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:57.947158    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wrj9" event={"ID":"0d6301d1-c237-48f6-b5af-921a4a4540f1","Type":"ContainerDied","Data":"acaddf93f8b9eec191d5161e147a6c2e0b6c4dbad5c5c90bd5182e20774f9f63"}
Jul 24 03:17:58.155084 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:58.155032    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:58.155084 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:58.155092    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:58.155287 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:58.155189    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:58.155287 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:58.155193    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:58.155287 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:58.155240    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs podName:6d26b109-0e00-4704-95a2-9cda7952facb nodeName:}" failed. No retries permitted until 2026-07-24 03:18:30.155228007 +0000 UTC m=+65.118321272 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs") pod "network-metrics-daemon-mp65w" (UID: "6d26b109-0e00-4704-95a2-9cda7952facb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 03:17:58.155287 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:58.155255    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret podName:3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d nodeName:}" failed. No retries permitted until 2026-07-24 03:18:30.155247527 +0000 UTC m=+65.118340793 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret") pod "global-pull-secret-syncer-6vl7h" (UID: "3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 03:17:58.356820 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:58.356726    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwfk5\" (UniqueName: \"kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5\") pod \"network-check-target-5g2fw\" (UID: \"7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0\") " pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:58.356958 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:58.356854    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 03:17:58.356958 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:58.356879    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 03:17:58.356958 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:58.356890    2574 projected.go:196] Error preparing data for projected volume kube-api-access-qwfk5 for pod openshift-network-diagnostics/network-check-target-5g2fw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:58.356958 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:58.356944    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5 podName:7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:30.356930438 +0000 UTC m=+65.320023703 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwfk5" (UniqueName: "kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5") pod "network-check-target-5g2fw" (UID: "7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 03:17:58.567253 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:58.567216    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:17:58.567253 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:58.567242    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:17:58.567511 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:17:58.567247    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:17:58.567511 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:58.567356    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5g2fw" podUID="7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0"
Jul 24 03:17:58.567628 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:58.567513    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6vl7h" podUID="3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d"
Jul 24 03:17:58.567628 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:17:58.567616    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:18:00.412856 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.412617    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-227.ec2.internal" event="NodeReady"
Jul 24 03:18:00.413341 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.412931    2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 03:18:00.446407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.446368    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74ffdd85b9-6cwrn"]
Jul 24 03:18:00.449385 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.449360    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.452015 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.451986    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 03:18:00.452177 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.452090    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 03:18:00.452407 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.452388    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 03:18:00.452530 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.452485    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bwrkh\""
Jul 24 03:18:00.456400 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.455664    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tb9dz"]
Jul 24 03:18:00.458709 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.458690    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:00.459519 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.459499    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 03:18:00.460444 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.460401    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-757zg"]
Jul 24 03:18:00.461078 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.461055    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 03:18:00.461171 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.461060    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fzmf8\""
Jul 24 03:18:00.461451 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.461414    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 03:18:00.463544 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.463525    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-757zg"
Jul 24 03:18:00.465598 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.465575    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 03:18:00.465953 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.465932    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74ffdd85b9-6cwrn"]
Jul 24 03:18:00.466584 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.466566    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 03:18:00.467160 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.467115    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k789z\""
Jul 24 03:18:00.467402 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.467383    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 03:18:00.470877 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.470853    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tb9dz"]
Jul 24 03:18:00.474878 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.474853    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-757zg"]
Jul 24 03:18:00.566916 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.566880    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:18:00.566916 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.566918    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:18:00.567166 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.566877    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:18:00.569260 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.569237    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 03:18:00.569510 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.569305    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 03:18:00.569637 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.569508    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-frl8x\""
Jul 24 03:18:00.569637 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.569549    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 03:18:00.569740 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.569721    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 03:18:00.569796 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.569772    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n6kfx\""
Jul 24 03:18:00.572832 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.572807    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d3b50f3-2285-4c78-9972-4964483c7837-trusted-ca\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.572939 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.572846    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d3b50f3-2285-4c78-9972-4964483c7837-image-registry-private-configuration\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.572939 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.572874    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d3b50f3-2285-4c78-9972-4964483c7837-installation-pull-secrets\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.573057 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.572939    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d10c276-6ba4-44b2-9895-1bb803660d4e-tmp-dir\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:00.573057 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.572987    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe775e9b-fb53-4c76-b95e-a43de1163af0-cert\") pod \"ingress-canary-757zg\" (UID: \"fe775e9b-fb53-4c76-b95e-a43de1163af0\") " pod="openshift-ingress-canary/ingress-canary-757zg"
Jul 24 03:18:00.573057 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.573044    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-bound-sa-token\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.573208 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.573133    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d10c276-6ba4-44b2-9895-1bb803660d4e-config-volume\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:00.573208 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.573169    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d3b50f3-2285-4c78-9972-4964483c7837-registry-certificates\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.573208 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.573196    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4rz4\" (UniqueName: \"kubernetes.io/projected/fe775e9b-fb53-4c76-b95e-a43de1163af0-kube-api-access-v4rz4\") pod \"ingress-canary-757zg\" (UID: \"fe775e9b-fb53-4c76-b95e-a43de1163af0\") " pod="openshift-ingress-canary/ingress-canary-757zg"
Jul 24 03:18:00.573356 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.573222    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:00.573356 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.573270    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf9sz\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-kube-api-access-qf9sz\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.573356 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.573295    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxwml\" (UniqueName: \"kubernetes.io/projected/3d10c276-6ba4-44b2-9895-1bb803660d4e-kube-api-access-dxwml\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:00.573356 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.573318    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.573529 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.573364    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d3b50f3-2285-4c78-9972-4964483c7837-ca-trust-extracted\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.674652 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.674558    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d3b50f3-2285-4c78-9972-4964483c7837-trusted-ca\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.674652 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.674597    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d3b50f3-2285-4c78-9972-4964483c7837-image-registry-private-configuration\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.674652 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.674620    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d3b50f3-2285-4c78-9972-4964483c7837-installation-pull-secrets\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.674652 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.674646    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d10c276-6ba4-44b2-9895-1bb803660d4e-tmp-dir\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:00.674932 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.674725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe775e9b-fb53-4c76-b95e-a43de1163af0-cert\") pod \"ingress-canary-757zg\" (UID: \"fe775e9b-fb53-4c76-b95e-a43de1163af0\") " pod="openshift-ingress-canary/ingress-canary-757zg"
Jul 24 03:18:00.674932 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.674802    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-bound-sa-token\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.674932 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.674868    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d10c276-6ba4-44b2-9895-1bb803660d4e-config-volume\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:00.674932 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.674923    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d3b50f3-2285-4c78-9972-4964483c7837-registry-certificates\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.675081 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.674955    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4rz4\" (UniqueName: \"kubernetes.io/projected/fe775e9b-fb53-4c76-b95e-a43de1163af0-kube-api-access-v4rz4\") pod \"ingress-canary-757zg\" (UID: \"fe775e9b-fb53-4c76-b95e-a43de1163af0\") " pod="openshift-ingress-canary/ingress-canary-757zg"
Jul 24 03:18:00.675081 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.674990    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:00.675081 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.675018    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf9sz\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-kube-api-access-qf9sz\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.675081 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.675045    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxwml\" (UniqueName: \"kubernetes.io/projected/3d10c276-6ba4-44b2-9895-1bb803660d4e-kube-api-access-dxwml\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:00.675081 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.675052    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d10c276-6ba4-44b2-9895-1bb803660d4e-tmp-dir\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:00.675081 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.675070    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.675289 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.675123    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d3b50f3-2285-4c78-9972-4964483c7837-ca-trust-extracted\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.675289 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:00.675161    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:18:00.675289 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:00.675176    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74ffdd85b9-6cwrn: secret "image-registry-tls" not found
Jul 24 03:18:00.675289 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:00.675245    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls podName:8d3b50f3-2285-4c78-9972-4964483c7837 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:01.175216418 +0000 UTC m=+36.138309692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls") pod "image-registry-74ffdd85b9-6cwrn" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837") : secret "image-registry-tls" not found
Jul 24 03:18:00.675575 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:00.675558    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:18:00.675665 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:00.675611    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls podName:3d10c276-6ba4-44b2-9895-1bb803660d4e nodeName:}" failed. No retries permitted until 2026-07-24 03:18:01.175594452 +0000 UTC m=+36.138687719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls") pod "dns-default-tb9dz" (UID: "3d10c276-6ba4-44b2-9895-1bb803660d4e") : secret "dns-default-metrics-tls" not found
Jul 24 03:18:00.676211 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.675860    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d3b50f3-2285-4c78-9972-4964483c7837-trusted-ca\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.676211 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.676038    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d3b50f3-2285-4c78-9972-4964483c7837-ca-trust-extracted\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.676211 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.676172    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d3b50f3-2285-4c78-9972-4964483c7837-registry-certificates\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.676211 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.676202    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d10c276-6ba4-44b2-9895-1bb803660d4e-config-volume\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:00.679817 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.679795    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d3b50f3-2285-4c78-9972-4964483c7837-installation-pull-secrets\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.679932 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.679911    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe775e9b-fb53-4c76-b95e-a43de1163af0-cert\") pod \"ingress-canary-757zg\" (UID: \"fe775e9b-fb53-4c76-b95e-a43de1163af0\") " pod="openshift-ingress-canary/ingress-canary-757zg"
Jul 24 03:18:00.679998 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.679927    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d3b50f3-2285-4c78-9972-4964483c7837-image-registry-private-configuration\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.684723 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.684672    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxwml\" (UniqueName: \"kubernetes.io/projected/3d10c276-6ba4-44b2-9895-1bb803660d4e-kube-api-access-dxwml\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:00.684942 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.684922    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-bound-sa-token\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.685009 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.684987    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf9sz\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-kube-api-access-qf9sz\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:00.685065 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.685046    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4rz4\" (UniqueName: \"kubernetes.io/projected/fe775e9b-fb53-4c76-b95e-a43de1163af0-kube-api-access-v4rz4\") pod \"ingress-canary-757zg\" (UID: \"fe775e9b-fb53-4c76-b95e-a43de1163af0\") " pod="openshift-ingress-canary/ingress-canary-757zg"
Jul 24 03:18:00.780163 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:00.780128    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-757zg"
Jul 24 03:18:01.180714 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:01.180666    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:01.180714 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:01.180717    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:01.180971 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:01.180874    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:18:01.180971 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:01.180878    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:18:01.180971 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:01.180966    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls podName:3d10c276-6ba4-44b2-9895-1bb803660d4e nodeName:}" failed. No retries permitted until 2026-07-24 03:18:02.180941146 +0000 UTC m=+37.144034432 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls") pod "dns-default-tb9dz" (UID: "3d10c276-6ba4-44b2-9895-1bb803660d4e") : secret "dns-default-metrics-tls" not found
Jul 24 03:18:01.180971 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:01.180888    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74ffdd85b9-6cwrn: secret "image-registry-tls" not found
Jul 24 03:18:01.181165 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:01.181028    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls podName:8d3b50f3-2285-4c78-9972-4964483c7837 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:02.181013665 +0000 UTC m=+37.144106950 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls") pod "image-registry-74ffdd85b9-6cwrn" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837") : secret "image-registry-tls" not found
Jul 24 03:18:02.188770 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:02.188715    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:02.188770 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:02.188771    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:02.189328 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:02.188876    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:18:02.189328 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:02.188890    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:18:02.189328 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:02.188901    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74ffdd85b9-6cwrn: secret "image-registry-tls" not found
Jul 24 03:18:02.189328 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:02.188949    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls podName:8d3b50f3-2285-4c78-9972-4964483c7837 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:04.188931366 +0000 UTC m=+39.152024630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls") pod "image-registry-74ffdd85b9-6cwrn" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837") : secret "image-registry-tls" not found
Jul 24 03:18:02.189328 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:02.188965    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls podName:3d10c276-6ba4-44b2-9895-1bb803660d4e nodeName:}" failed. No retries permitted until 2026-07-24 03:18:04.188957285 +0000 UTC m=+39.152050550 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls") pod "dns-default-tb9dz" (UID: "3d10c276-6ba4-44b2-9895-1bb803660d4e") : secret "dns-default-metrics-tls" not found
Jul 24 03:18:02.523673 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:02.523636    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-757zg"]
Jul 24 03:18:02.585643 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:18:02.585605    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe775e9b_fb53_4c76_b95e_a43de1163af0.slice/crio-816243bca4513d2e562054bac2bc768ff5f7287c086cf0b545303819f5277ff3 WatchSource:0}: Error finding container 816243bca4513d2e562054bac2bc768ff5f7287c086cf0b545303819f5277ff3: Status 404 returned error can't find the container with id 816243bca4513d2e562054bac2bc768ff5f7287c086cf0b545303819f5277ff3
Jul 24 03:18:02.961287 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:02.961056    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-757zg" event={"ID":"fe775e9b-fb53-4c76-b95e-a43de1163af0","Type":"ContainerStarted","Data":"816243bca4513d2e562054bac2bc768ff5f7287c086cf0b545303819f5277ff3"}
Jul 24 03:18:02.963892 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:02.963865    2574 generic.go:356] "Generic (PLEG): container finished" podID="0d6301d1-c237-48f6-b5af-921a4a4540f1" containerID="03f772e61ddd9b14a4994bb7ab55808f39fd1ebac7ca6672e1df0d5df1a45402" exitCode=0
Jul 24 03:18:02.964021 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:02.963909    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wrj9" event={"ID":"0d6301d1-c237-48f6-b5af-921a4a4540f1","Type":"ContainerDied","Data":"03f772e61ddd9b14a4994bb7ab55808f39fd1ebac7ca6672e1df0d5df1a45402"}
Jul 24 03:18:03.971137 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:03.971103    2574 generic.go:356] "Generic (PLEG): container finished" podID="0d6301d1-c237-48f6-b5af-921a4a4540f1" containerID="3623bf0e030c9ef9426561e5a8a7e85b58ff8eff548dc77d639b1a24a86c5f56" exitCode=0
Jul 24 03:18:03.971869 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:03.971144    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wrj9" event={"ID":"0d6301d1-c237-48f6-b5af-921a4a4540f1","Type":"ContainerDied","Data":"3623bf0e030c9ef9426561e5a8a7e85b58ff8eff548dc77d639b1a24a86c5f56"}
Jul 24 03:18:04.207374 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:04.207335    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:04.207576 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:04.207399    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:04.207576 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:04.207511    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:18:04.207576 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:04.207536    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:18:04.207576 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:04.207550    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74ffdd85b9-6cwrn: secret "image-registry-tls" not found
Jul 24 03:18:04.208241 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:04.208217    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls podName:3d10c276-6ba4-44b2-9895-1bb803660d4e nodeName:}" failed. No retries permitted until 2026-07-24 03:18:08.20819394 +0000 UTC m=+43.171287209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls") pod "dns-default-tb9dz" (UID: "3d10c276-6ba4-44b2-9895-1bb803660d4e") : secret "dns-default-metrics-tls" not found
Jul 24 03:18:04.208363 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:04.208251    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls podName:8d3b50f3-2285-4c78-9972-4964483c7837 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:08.208238989 +0000 UTC m=+43.171332270 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls") pod "image-registry-74ffdd85b9-6cwrn" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837") : secret "image-registry-tls" not found
Jul 24 03:18:04.976681 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:04.976648    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wrj9" event={"ID":"0d6301d1-c237-48f6-b5af-921a4a4540f1","Type":"ContainerStarted","Data":"9498f4740791a8a8f100bac5a7481ef0e7805f4bdccf37ca6dd31152fc8d60be"}
Jul 24 03:18:04.998994 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:04.998811    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4wrj9" podStartSLOduration=6.105339446 podStartE2EDuration="38.998789663s" totalImagesPullingTime="32.893450217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:04.997438201 +0000 UTC m=+39.960531478" watchObservedRunningTime="2026-07-24 03:18:04.998789663 +0000 UTC m=+39.961882951"
Jul 24 03:18:05.980974 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:05.980927    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-757zg" event={"ID":"fe775e9b-fb53-4c76-b95e-a43de1163af0","Type":"ContainerStarted","Data":"4961dc012a389636af7db9a7db62b8ec40085a0993e4036283f7961a9d862b73"}
Jul 24 03:18:05.996239 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:05.996187    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-757zg" podStartSLOduration=3.332500265 podStartE2EDuration="5.996169484s" totalImagesPullingTime="2.663669219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:05.995465142 +0000 UTC m=+40.958558429" watchObservedRunningTime="2026-07-24 03:18:05.996169484 +0000 UTC m=+40.959262776"
Jul 24 03:18:08.239540 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:08.239479    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:08.239540 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:08.239537    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:08.239966 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:08.239638    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:18:08.239966 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:08.239673    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:18:08.239966 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:08.239683    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74ffdd85b9-6cwrn: secret "image-registry-tls" not found
Jul 24 03:18:08.239966 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:08.239708    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls podName:3d10c276-6ba4-44b2-9895-1bb803660d4e nodeName:}" failed. No retries permitted until 2026-07-24 03:18:16.239691709 +0000 UTC m=+51.202784973 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls") pod "dns-default-tb9dz" (UID: "3d10c276-6ba4-44b2-9895-1bb803660d4e") : secret "dns-default-metrics-tls" not found
Jul 24 03:18:08.239966 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:08.239733    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls podName:8d3b50f3-2285-4c78-9972-4964483c7837 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:16.239721991 +0000 UTC m=+51.202815256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls") pod "image-registry-74ffdd85b9-6cwrn" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837") : secret "image-registry-tls" not found
Jul 24 03:18:16.302537 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:16.302497    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:16.302537 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:16.302540    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:16.303019 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:16.302662    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:18:16.303019 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:16.302666    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:18:16.303019 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:16.302683    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74ffdd85b9-6cwrn: secret "image-registry-tls" not found
Jul 24 03:18:16.303019 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:16.302736    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls podName:8d3b50f3-2285-4c78-9972-4964483c7837 nodeName:}" failed. No retries permitted until 2026-07-24 03:18:32.302714107 +0000 UTC m=+67.265807384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls") pod "image-registry-74ffdd85b9-6cwrn" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837") : secret "image-registry-tls" not found
Jul 24 03:18:16.303019 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:16.302752    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls podName:3d10c276-6ba4-44b2-9895-1bb803660d4e nodeName:}" failed. No retries permitted until 2026-07-24 03:18:32.302743817 +0000 UTC m=+67.265837081 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls") pod "dns-default-tb9dz" (UID: "3d10c276-6ba4-44b2-9895-1bb803660d4e") : secret "dns-default-metrics-tls" not found
Jul 24 03:18:25.952724 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:25.952693    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zgfsh"
Jul 24 03:18:30.215468 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.215383    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:18:30.215870 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.215486    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:18:30.217789 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.217770    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 03:18:30.217835 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.217816    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 03:18:30.226682 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:30.226655    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 03:18:30.226744 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:30.226734    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs podName:6d26b109-0e00-4704-95a2-9cda7952facb nodeName:}" failed. No retries permitted until 2026-07-24 03:19:34.226717666 +0000 UTC m=+129.189810932 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs") pod "network-metrics-daemon-mp65w" (UID: "6d26b109-0e00-4704-95a2-9cda7952facb") : secret "metrics-daemon-secret" not found
Jul 24 03:18:30.230482 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.230459    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d-original-pull-secret\") pod \"global-pull-secret-syncer-6vl7h\" (UID: \"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d\") " pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:18:30.291876 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.291824    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6vl7h"
Jul 24 03:18:30.417175 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.417142    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwfk5\" (UniqueName: \"kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5\") pod \"network-check-target-5g2fw\" (UID: \"7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0\") " pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:18:30.419503 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.419475    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6vl7h"]
Jul 24 03:18:30.419669 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.419650    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 03:18:30.423719 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:18:30.423686    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d93655f_1e04_43cc_ac4b_1cd6fb0a9a1d.slice/crio-698ff3da254d0081dd0ca611e0efe89df09d94189e5f7061053ecf4148a0b692 WatchSource:0}: Error finding container 698ff3da254d0081dd0ca611e0efe89df09d94189e5f7061053ecf4148a0b692: Status 404 returned error can't find the container with id 698ff3da254d0081dd0ca611e0efe89df09d94189e5f7061053ecf4148a0b692
Jul 24 03:18:30.429868 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.429848    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 03:18:30.440883 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.440863    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwfk5\" (UniqueName: \"kubernetes.io/projected/7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0-kube-api-access-qwfk5\") pod \"network-check-target-5g2fw\" (UID: \"7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0\") " pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:18:30.588378 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.588302    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n6kfx\""
Jul 24 03:18:30.597295 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.597265    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:18:30.743740 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:30.743701    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5g2fw"]
Jul 24 03:18:30.747196 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:18:30.747171    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fe66cfe_8b17_46a2_baa7_ebe11afa9fe0.slice/crio-e42fdfe194118d6e2800389481c259ca7c79a8463ed4b139fcee22e0932799fa WatchSource:0}: Error finding container e42fdfe194118d6e2800389481c259ca7c79a8463ed4b139fcee22e0932799fa: Status 404 returned error can't find the container with id e42fdfe194118d6e2800389481c259ca7c79a8463ed4b139fcee22e0932799fa
Jul 24 03:18:31.041976 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:31.041935    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5g2fw" event={"ID":"7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0","Type":"ContainerStarted","Data":"e42fdfe194118d6e2800389481c259ca7c79a8463ed4b139fcee22e0932799fa"}
Jul 24 03:18:31.043173 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:31.043149    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6vl7h" event={"ID":"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d","Type":"ContainerStarted","Data":"698ff3da254d0081dd0ca611e0efe89df09d94189e5f7061053ecf4148a0b692"}
Jul 24 03:18:32.333360 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:32.333321    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:18:32.333360 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:32.333366    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:18:32.333925 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:32.333524    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:18:32.333925 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:32.333539    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74ffdd85b9-6cwrn: secret "image-registry-tls" not found
Jul 24 03:18:32.333925 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:32.333602    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls podName:8d3b50f3-2285-4c78-9972-4964483c7837 nodeName:}" failed. No retries permitted until 2026-07-24 03:19:04.333581867 +0000 UTC m=+99.296675146 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls") pod "image-registry-74ffdd85b9-6cwrn" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837") : secret "image-registry-tls" not found
Jul 24 03:18:32.334093 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:32.334003    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:18:32.334093 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:18:32.334038    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls podName:3d10c276-6ba4-44b2-9895-1bb803660d4e nodeName:}" failed. No retries permitted until 2026-07-24 03:19:04.334027397 +0000 UTC m=+99.297120675 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls") pod "dns-default-tb9dz" (UID: "3d10c276-6ba4-44b2-9895-1bb803660d4e") : secret "dns-default-metrics-tls" not found
Jul 24 03:18:36.056659 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:36.056602    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6vl7h" event={"ID":"3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d","Type":"ContainerStarted","Data":"11efc2d59a85258a7c4c5a26104750f47cd8d9ec27000a06ccb2a12ca327f78d"}
Jul 24 03:18:36.058197 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:36.058170    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5g2fw" event={"ID":"7fe66cfe-8b17-46a2-baa7-ebe11afa9fe0","Type":"ContainerStarted","Data":"bca4f4d7b85cd7cf5bc2274adef3d00464bc2f0c9d87f5372e15c08737aa4cb3"}
Jul 24 03:18:36.058322 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:36.058308    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:18:36.070811 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:36.070750    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6vl7h" podStartSLOduration=65.277210265 podStartE2EDuration="1m10.070735985s" totalImagesPullingTime="4.79352572s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:36.069862384 +0000 UTC m=+71.032955683" watchObservedRunningTime="2026-07-24 03:18:36.070735985 +0000 UTC m=+71.033829271"
Jul 24 03:18:36.083242 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:18:36.083197    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5g2fw" podStartSLOduration=65.61762779 podStartE2EDuration="1m10.083181309s" totalImagesPullingTime="4.465553519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:18:36.082374315 +0000 UTC m=+71.045467593" watchObservedRunningTime="2026-07-24 03:18:36.083181309 +0000 UTC m=+71.046274595"
Jul 24 03:19:04.371572 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:19:04.371535    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:19:04.372120 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:19:04.371581    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:19:04.372120 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:19:04.371699    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:19:04.372120 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:19:04.371728    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:19:04.372120 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:19:04.371742    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74ffdd85b9-6cwrn: secret "image-registry-tls" not found
Jul 24 03:19:04.372120 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:19:04.371767    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls podName:3d10c276-6ba4-44b2-9895-1bb803660d4e nodeName:}" failed. No retries permitted until 2026-07-24 03:20:08.37174905 +0000 UTC m=+163.334842315 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls") pod "dns-default-tb9dz" (UID: "3d10c276-6ba4-44b2-9895-1bb803660d4e") : secret "dns-default-metrics-tls" not found
Jul 24 03:19:04.372120 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:19:04.371804    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls podName:8d3b50f3-2285-4c78-9972-4964483c7837 nodeName:}" failed. No retries permitted until 2026-07-24 03:20:08.371787719 +0000 UTC m=+163.334880987 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls") pod "image-registry-74ffdd85b9-6cwrn" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837") : secret "image-registry-tls" not found
Jul 24 03:19:07.063245 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:19:07.063206    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5g2fw"
Jul 24 03:19:34.284948 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:19:34.284893    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:19:34.285486 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:19:34.285056    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 03:19:34.285486 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:19:34.285132    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs podName:6d26b109-0e00-4704-95a2-9cda7952facb nodeName:}" failed. No retries permitted until 2026-07-24 03:21:36.285114856 +0000 UTC m=+251.248208120 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs") pod "network-metrics-daemon-mp65w" (UID: "6d26b109-0e00-4704-95a2-9cda7952facb") : secret "metrics-daemon-secret" not found
Jul 24 03:20:03.461863 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:20:03.461807    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn" podUID="8d3b50f3-2285-4c78-9972-4964483c7837"
Jul 24 03:20:03.473022 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:20:03.472975    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tb9dz" podUID="3d10c276-6ba4-44b2-9895-1bb803660d4e"
Jul 24 03:20:03.578170 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:20:03.578118    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mp65w" podUID="6d26b109-0e00-4704-95a2-9cda7952facb"
Jul 24 03:20:04.274324 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:20:04.274292    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:20:04.274324 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:20:04.274330    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:20:08.416354 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:20:08.416295    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:20:08.416354 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:20:08.416356    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls\") pod \"image-registry-74ffdd85b9-6cwrn\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") " pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:20:08.416787 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:20:08.416498    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 03:20:08.416787 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:20:08.416501    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 03:20:08.416787 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:20:08.416510    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74ffdd85b9-6cwrn: secret "image-registry-tls" not found
Jul 24 03:20:08.416787 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:20:08.416565    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls podName:8d3b50f3-2285-4c78-9972-4964483c7837 nodeName:}" failed. No retries permitted until 2026-07-24 03:22:10.416551304 +0000 UTC m=+285.379644570 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls") pod "image-registry-74ffdd85b9-6cwrn" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837") : secret "image-registry-tls" not found
Jul 24 03:20:08.416787 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:20:08.416576    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls podName:3d10c276-6ba4-44b2-9895-1bb803660d4e nodeName:}" failed. No retries permitted until 2026-07-24 03:22:10.416571246 +0000 UTC m=+285.379664510 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls") pod "dns-default-tb9dz" (UID: "3d10c276-6ba4-44b2-9895-1bb803660d4e") : secret "dns-default-metrics-tls" not found
Jul 24 03:20:15.567745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:20:15.567704    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:21:18.708770 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.708731    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"]
Jul 24 03:21:18.710385 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.710369    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"
Jul 24 03:21:18.712316 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.712281    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xbwck"]
Jul 24 03:21:18.713309 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.713287    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 24 03:21:18.713740 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.713635    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 24 03:21:18.713914 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.713885    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 24 03:21:18.713980 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.713960    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jk6sz\""
Jul 24 03:21:18.714034 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.714009    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 24 03:21:18.714262 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.714246    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-m7lml"]
Jul 24 03:21:18.714360 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.714346    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xbwck"
Jul 24 03:21:18.715804 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.715778    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-bjpkp"]
Jul 24 03:21:18.715944 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.715927    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-m7lml"
Jul 24 03:21:18.716615 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.716596    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 24 03:21:18.716728 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.716709    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 24 03:21:18.716728 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.716721    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-dpl4z\""
Jul 24 03:21:18.717309 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.717292    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-799b44d47c-5c54q"]
Jul 24 03:21:18.717412 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.717398    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:18.718256 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.718232    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-j5sfj\""
Jul 24 03:21:18.718821 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.718803    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:18.719550 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.719533    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 24 03:21:18.720175 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.720160    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 24 03:21:18.721813 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.721789    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 24 03:21:18.721920 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.721833    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 24 03:21:18.721920 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.721852    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 03:21:18.721920 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.721880    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 24 03:21:18.722076 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.722009    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 24 03:21:18.722076 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.722066    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 24 03:21:18.722173 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.722109    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-6dv9x\""
Jul 24 03:21:18.722173 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.722014    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 03:21:18.722443 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.722410    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-288bb\""
Jul 24 03:21:18.722443 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.722405    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 24 03:21:18.733650 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.733189    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 24 03:21:18.733650 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.733538    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"]
Jul 24 03:21:18.734592 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.734566    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-m7lml"]
Jul 24 03:21:18.735960 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.735734    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xbwck"]
Jul 24 03:21:18.742818 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.742795    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-bjpkp"]
Jul 24 03:21:18.749260 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.749238    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-799b44d47c-5c54q"]
Jul 24 03:21:18.811399 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.811356    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"]
Jul 24 03:21:18.812323 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812303    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/69510a47-b326-4920-acb8-1baa907c926d-config\") pod \"console-operator-584dcb6b54-bjpkp\" (UID: \"69510a47-b326-4920-acb8-1baa907c926d\") " pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:18.812381 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812333    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-default-certificate\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:18.812381 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812365    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc7cb98c-bd37-44bf-80f0-1cd615e8a36f-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp\" (UID: \"bc7cb98c-bd37-44bf-80f0-1cd615e8a36f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"
Jul 24 03:21:18.812475 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812390    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpdcl\" (UniqueName: \"kubernetes.io/projected/09f9b961-e743-4da0-be0d-80a58f646b62-kube-api-access-bpdcl\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:18.812475 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812405    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/69510a47-b326-4920-acb8-1baa907c926d-serving-cert\") pod \"console-operator-584dcb6b54-bjpkp\" (UID: \"69510a47-b326-4920-acb8-1baa907c926d\") " pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:18.812475 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812420    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc7cb98c-bd37-44bf-80f0-1cd615e8a36f-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp\" (UID: \"bc7cb98c-bd37-44bf-80f0-1cd615e8a36f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"
Jul 24 03:21:18.812579 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812485    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:18.812579 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812510    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69510a47-b326-4920-acb8-1baa907c926d-trusted-ca\") pod \"console-operator-584dcb6b54-bjpkp\" (UID: \"69510a47-b326-4920-acb8-1baa907c926d\") " pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:18.812657 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812576    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qhtt\" (UniqueName: \"kubernetes.io/projected/3af1a82c-c889-4da3-95fa-6812a25bcea3-kube-api-access-8qhtt\") pod \"volume-data-source-validator-549c6d54cf-xbwck\" (UID: \"3af1a82c-c889-4da3-95fa-6812a25bcea3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xbwck"
Jul 24 03:21:18.812657 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812600    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-stats-auth\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:18.812657 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812626    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:18.812756 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812653    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rljmq\" (UniqueName: \"kubernetes.io/projected/bc7cb98c-bd37-44bf-80f0-1cd615e8a36f-kube-api-access-rljmq\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp\" (UID: \"bc7cb98c-bd37-44bf-80f0-1cd615e8a36f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"
Jul 24 03:21:18.812756 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812683    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl8ll\" (UniqueName: \"kubernetes.io/projected/c732f138-0f8d-4ab9-b0b8-a2581b916fa9-kube-api-access-nl8ll\") pod \"network-check-source-7d88cb55d8-m7lml\" (UID: \"c732f138-0f8d-4ab9-b0b8-a2581b916fa9\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-m7lml"
Jul 24 03:21:18.812756 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.812698    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64d6f\" (UniqueName: \"kubernetes.io/projected/69510a47-b326-4920-acb8-1baa907c926d-kube-api-access-64d6f\") pod \"console-operator-584dcb6b54-bjpkp\" (UID: \"69510a47-b326-4920-acb8-1baa907c926d\") " pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:18.813215 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.813200    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"]
Jul 24 03:21:18.813311 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.813299    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"
Jul 24 03:21:18.814953 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.814937    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:18.816165 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.816147    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 24 03:21:18.816394 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.816381    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 24 03:21:18.816675 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.816656    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 24 03:21:18.816767 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.816719    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 03:21:18.816937 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.816919    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-cgfwq\""
Jul 24 03:21:18.817022 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.817004    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 24 03:21:18.817123 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.817110    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 03:21:18.817213 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.817122    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2s5n9\""
Jul 24 03:21:18.817711 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.817695    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 24 03:21:18.828283 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.828259    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"]
Jul 24 03:21:18.829139 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.829119    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"]
Jul 24 03:21:18.913305 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913266    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qhtt\" (UniqueName: \"kubernetes.io/projected/3af1a82c-c889-4da3-95fa-6812a25bcea3-kube-api-access-8qhtt\") pod \"volume-data-source-validator-549c6d54cf-xbwck\" (UID: \"3af1a82c-c889-4da3-95fa-6812a25bcea3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xbwck"
Jul 24 03:21:18.913305 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913304    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-stats-auth\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:18.913609 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913326    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:18.913609 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913346    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rljmq\" (UniqueName: \"kubernetes.io/projected/bc7cb98c-bd37-44bf-80f0-1cd615e8a36f-kube-api-access-rljmq\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp\" (UID: \"bc7cb98c-bd37-44bf-80f0-1cd615e8a36f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"
Jul 24 03:21:18.913609 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913376    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rfjp\" (UniqueName: \"kubernetes.io/projected/136cde74-b9d8-4c3c-9bca-74b7628db11b-kube-api-access-7rfjp\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:18.913609 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913409    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl8ll\" (UniqueName: \"kubernetes.io/projected/c732f138-0f8d-4ab9-b0b8-a2581b916fa9-kube-api-access-nl8ll\") pod \"network-check-source-7d88cb55d8-m7lml\" (UID: \"c732f138-0f8d-4ab9-b0b8-a2581b916fa9\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-m7lml"
Jul 24 03:21:18.913609 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913458    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64d6f\" (UniqueName: \"kubernetes.io/projected/69510a47-b326-4920-acb8-1baa907c926d-kube-api-access-64d6f\") pod \"console-operator-584dcb6b54-bjpkp\" (UID: \"69510a47-b326-4920-acb8-1baa907c926d\") " pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:18.913609 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:18.913513    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle podName:09f9b961-e743-4da0-be0d-80a58f646b62 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:19.413490496 +0000 UTC m=+234.376583779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle") pod "router-default-799b44d47c-5c54q" (UID: "09f9b961-e743-4da0-be0d-80a58f646b62") : configmap references non-existent config key: service-ca.crt
Jul 24 03:21:18.913609 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913551    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/69510a47-b326-4920-acb8-1baa907c926d-config\") pod \"console-operator-584dcb6b54-bjpkp\" (UID: \"69510a47-b326-4920-acb8-1baa907c926d\") " pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:18.913609 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913581    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/136cde74-b9d8-4c3c-9bca-74b7628db11b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:18.913999 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913612    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-default-certificate\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:18.913999 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913666    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc7cb98c-bd37-44bf-80f0-1cd615e8a36f-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp\" (UID: \"bc7cb98c-bd37-44bf-80f0-1cd615e8a36f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"
Jul 24 03:21:18.913999 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913696    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:18.913999 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913732    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpdcl\" (UniqueName: \"kubernetes.io/projected/09f9b961-e743-4da0-be0d-80a58f646b62-kube-api-access-bpdcl\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:18.913999 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913758    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/69510a47-b326-4920-acb8-1baa907c926d-serving-cert\") pod \"console-operator-584dcb6b54-bjpkp\" (UID: \"69510a47-b326-4920-acb8-1baa907c926d\") " pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:18.913999 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913786    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc7cb98c-bd37-44bf-80f0-1cd615e8a36f-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp\" (UID: \"bc7cb98c-bd37-44bf-80f0-1cd615e8a36f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"
Jul 24 03:21:18.913999 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913810    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngx4s\" (UniqueName: \"kubernetes.io/projected/697e431d-f03f-4ddb-b7a5-fadefc9b034f-kube-api-access-ngx4s\") pod \"cluster-samples-operator-6748489656-mb4q7\" (UID: \"697e431d-f03f-4ddb-b7a5-fadefc9b034f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"
Jul 24 03:21:18.913999 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913848    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:18.913999 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913886    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mb4q7\" (UID: \"697e431d-f03f-4ddb-b7a5-fadefc9b034f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"
Jul 24 03:21:18.913999 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.913913    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69510a47-b326-4920-acb8-1baa907c926d-trusted-ca\") pod \"console-operator-584dcb6b54-bjpkp\" (UID: \"69510a47-b326-4920-acb8-1baa907c926d\") " pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:18.914752 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.914396    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/69510a47-b326-4920-acb8-1baa907c926d-config\") pod \"console-operator-584dcb6b54-bjpkp\" (UID: \"69510a47-b326-4920-acb8-1baa907c926d\") " pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:18.914752 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:18.914527    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 03:21:18.914752 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:18.914595    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs podName:09f9b961-e743-4da0-be0d-80a58f646b62 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:19.414577448 +0000 UTC m=+234.377670715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs") pod "router-default-799b44d47c-5c54q" (UID: "09f9b961-e743-4da0-be0d-80a58f646b62") : secret "router-metrics-certs-default" not found
Jul 24 03:21:18.914752 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.914731    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc7cb98c-bd37-44bf-80f0-1cd615e8a36f-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp\" (UID: \"bc7cb98c-bd37-44bf-80f0-1cd615e8a36f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"
Jul 24 03:21:18.914752 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.914749    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69510a47-b326-4920-acb8-1baa907c926d-trusted-ca\") pod \"console-operator-584dcb6b54-bjpkp\" (UID: \"69510a47-b326-4920-acb8-1baa907c926d\") " pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:18.916234 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.916211    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc7cb98c-bd37-44bf-80f0-1cd615e8a36f-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp\" (UID: \"bc7cb98c-bd37-44bf-80f0-1cd615e8a36f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"
Jul 24 03:21:18.916234 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.916227    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-stats-auth\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:18.916562 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.916544    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/69510a47-b326-4920-acb8-1baa907c926d-serving-cert\") pod \"console-operator-584dcb6b54-bjpkp\" (UID: \"69510a47-b326-4920-acb8-1baa907c926d\") " pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:18.916694 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.916676    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-default-certificate\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:18.921921 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.921862    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rljmq\" (UniqueName: \"kubernetes.io/projected/bc7cb98c-bd37-44bf-80f0-1cd615e8a36f-kube-api-access-rljmq\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp\" (UID: \"bc7cb98c-bd37-44bf-80f0-1cd615e8a36f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"
Jul 24 03:21:18.922730 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.922692    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qhtt\" (UniqueName: \"kubernetes.io/projected/3af1a82c-c889-4da3-95fa-6812a25bcea3-kube-api-access-8qhtt\") pod \"volume-data-source-validator-549c6d54cf-xbwck\" (UID: \"3af1a82c-c889-4da3-95fa-6812a25bcea3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xbwck"
Jul 24 03:21:18.923443 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.923400    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl8ll\" (UniqueName: \"kubernetes.io/projected/c732f138-0f8d-4ab9-b0b8-a2581b916fa9-kube-api-access-nl8ll\") pod \"network-check-source-7d88cb55d8-m7lml\" (UID: \"c732f138-0f8d-4ab9-b0b8-a2581b916fa9\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-m7lml"
Jul 24 03:21:18.923766 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.923745    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64d6f\" (UniqueName: \"kubernetes.io/projected/69510a47-b326-4920-acb8-1baa907c926d-kube-api-access-64d6f\") pod \"console-operator-584dcb6b54-bjpkp\" (UID: \"69510a47-b326-4920-acb8-1baa907c926d\") " pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:18.923987 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:18.923969    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpdcl\" (UniqueName: \"kubernetes.io/projected/09f9b961-e743-4da0-be0d-80a58f646b62-kube-api-access-bpdcl\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:19.014613 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.014512    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:19.014613 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.014562    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngx4s\" (UniqueName: \"kubernetes.io/projected/697e431d-f03f-4ddb-b7a5-fadefc9b034f-kube-api-access-ngx4s\") pod \"cluster-samples-operator-6748489656-mb4q7\" (UID: \"697e431d-f03f-4ddb-b7a5-fadefc9b034f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"
Jul 24 03:21:19.014613 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.014600    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mb4q7\" (UID: \"697e431d-f03f-4ddb-b7a5-fadefc9b034f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"
Jul 24 03:21:19.014885 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.014647    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rfjp\" (UniqueName: \"kubernetes.io/projected/136cde74-b9d8-4c3c-9bca-74b7628db11b-kube-api-access-7rfjp\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:19.014885 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.014670    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/136cde74-b9d8-4c3c-9bca-74b7628db11b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:19.014885 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:19.014673    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 03:21:19.014885 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:19.014741    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls podName:136cde74-b9d8-4c3c-9bca-74b7628db11b nodeName:}" failed. No retries permitted until 2026-07-24 03:21:19.514719361 +0000 UTC m=+234.477812632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-p5ghk" (UID: "136cde74-b9d8-4c3c-9bca-74b7628db11b") : secret "cluster-monitoring-operator-tls" not found
Jul 24 03:21:19.014885 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:19.014751    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 03:21:19.014885 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:19.014814    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls podName:697e431d-f03f-4ddb-b7a5-fadefc9b034f nodeName:}" failed. No retries permitted until 2026-07-24 03:21:19.514795546 +0000 UTC m=+234.477888814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls") pod "cluster-samples-operator-6748489656-mb4q7" (UID: "697e431d-f03f-4ddb-b7a5-fadefc9b034f") : secret "samples-operator-tls" not found
Jul 24 03:21:19.015334 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.015314    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/136cde74-b9d8-4c3c-9bca-74b7628db11b-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:19.021414 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.021387    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"
Jul 24 03:21:19.025237 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.025211    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngx4s\" (UniqueName: \"kubernetes.io/projected/697e431d-f03f-4ddb-b7a5-fadefc9b034f-kube-api-access-ngx4s\") pod \"cluster-samples-operator-6748489656-mb4q7\" (UID: \"697e431d-f03f-4ddb-b7a5-fadefc9b034f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"
Jul 24 03:21:19.025341 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.025212    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rfjp\" (UniqueName: \"kubernetes.io/projected/136cde74-b9d8-4c3c-9bca-74b7628db11b-kube-api-access-7rfjp\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:19.032377 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.032356    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xbwck"
Jul 24 03:21:19.038662 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.038639    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-m7lml"
Jul 24 03:21:19.045940 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.045909    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:19.185046 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.184877    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp"]
Jul 24 03:21:19.188588 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:21:19.188540    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc7cb98c_bd37_44bf_80f0_1cd615e8a36f.slice/crio-7b19d9fe27369054249e55a19887a79ec049699e92b9e125ca540d65ec49ec7f WatchSource:0}: Error finding container 7b19d9fe27369054249e55a19887a79ec049699e92b9e125ca540d65ec49ec7f: Status 404 returned error can't find the container with id 7b19d9fe27369054249e55a19887a79ec049699e92b9e125ca540d65ec49ec7f
Jul 24 03:21:19.236698 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.236665    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-bjpkp"]
Jul 24 03:21:19.239905 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:21:19.239873    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69510a47_b326_4920_acb8_1baa907c926d.slice/crio-6f040a8261468f08af4a16d4d1b64e2693fe9bf52b342d6e8b9fd2b8975d4501 WatchSource:0}: Error finding container 6f040a8261468f08af4a16d4d1b64e2693fe9bf52b342d6e8b9fd2b8975d4501: Status 404 returned error can't find the container with id 6f040a8261468f08af4a16d4d1b64e2693fe9bf52b342d6e8b9fd2b8975d4501
Jul 24 03:21:19.408142 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.408099    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-m7lml"]
Jul 24 03:21:19.411614 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.411590    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xbwck"]
Jul 24 03:21:19.413360 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:21:19.413328    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc732f138_0f8d_4ab9_b0b8_a2581b916fa9.slice/crio-3d8f698b84fadc89de82d98f469c639e0cf16b2e3213c96e6e94488595b65f2c WatchSource:0}: Error finding container 3d8f698b84fadc89de82d98f469c639e0cf16b2e3213c96e6e94488595b65f2c: Status 404 returned error can't find the container with id 3d8f698b84fadc89de82d98f469c639e0cf16b2e3213c96e6e94488595b65f2c
Jul 24 03:21:19.417920 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.417784    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:19.417920 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.417886    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:19.418039 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:19.417889    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 03:21:19.418039 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:19.417970    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle podName:09f9b961-e743-4da0-be0d-80a58f646b62 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:20.417951815 +0000 UTC m=+235.381045097 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle") pod "router-default-799b44d47c-5c54q" (UID: "09f9b961-e743-4da0-be0d-80a58f646b62") : configmap references non-existent config key: service-ca.crt
Jul 24 03:21:19.418039 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:19.418003    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs podName:09f9b961-e743-4da0-be0d-80a58f646b62 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:20.417985714 +0000 UTC m=+235.381079028 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs") pod "router-default-799b44d47c-5c54q" (UID: "09f9b961-e743-4da0-be0d-80a58f646b62") : secret "router-metrics-certs-default" not found
Jul 24 03:21:19.457697 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.457668    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-m7lml" event={"ID":"c732f138-0f8d-4ab9-b0b8-a2581b916fa9","Type":"ContainerStarted","Data":"3d8f698b84fadc89de82d98f469c639e0cf16b2e3213c96e6e94488595b65f2c"}
Jul 24 03:21:19.458897 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.458874    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp" event={"ID":"bc7cb98c-bd37-44bf-80f0-1cd615e8a36f","Type":"ContainerStarted","Data":"7b19d9fe27369054249e55a19887a79ec049699e92b9e125ca540d65ec49ec7f"}
Jul 24 03:21:19.459913 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.459889    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp" event={"ID":"69510a47-b326-4920-acb8-1baa907c926d","Type":"ContainerStarted","Data":"6f040a8261468f08af4a16d4d1b64e2693fe9bf52b342d6e8b9fd2b8975d4501"}
Jul 24 03:21:19.460805 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.460785    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xbwck" event={"ID":"3af1a82c-c889-4da3-95fa-6812a25bcea3","Type":"ContainerStarted","Data":"255b552869c4d218cd10af6f59f09720b05f9ad59b0b6b3f7698a25c2e9e830f"}
Jul 24 03:21:19.518310 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.518273    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:19.518520 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:19.518334    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mb4q7\" (UID: \"697e431d-f03f-4ddb-b7a5-fadefc9b034f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"
Jul 24 03:21:19.518520 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:19.518470    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 03:21:19.518520 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:19.518471    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 03:21:19.518641 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:19.518526    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls podName:697e431d-f03f-4ddb-b7a5-fadefc9b034f nodeName:}" failed. No retries permitted until 2026-07-24 03:21:20.518511656 +0000 UTC m=+235.481604920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls") pod "cluster-samples-operator-6748489656-mb4q7" (UID: "697e431d-f03f-4ddb-b7a5-fadefc9b034f") : secret "samples-operator-tls" not found
Jul 24 03:21:19.518641 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:19.518539    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls podName:136cde74-b9d8-4c3c-9bca-74b7628db11b nodeName:}" failed. No retries permitted until 2026-07-24 03:21:20.518533561 +0000 UTC m=+235.481626825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-p5ghk" (UID: "136cde74-b9d8-4c3c-9bca-74b7628db11b") : secret "cluster-monitoring-operator-tls" not found
Jul 24 03:21:20.426130 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:20.426083    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:20.426638 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:20.426222    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle podName:09f9b961-e743-4da0-be0d-80a58f646b62 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:22.426198463 +0000 UTC m=+237.389291734 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle") pod "router-default-799b44d47c-5c54q" (UID: "09f9b961-e743-4da0-be0d-80a58f646b62") : configmap references non-existent config key: service-ca.crt
Jul 24 03:21:20.426638 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:20.426266    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:20.426638 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:20.426419    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 03:21:20.426638 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:20.426519    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs podName:09f9b961-e743-4da0-be0d-80a58f646b62 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:22.4265052 +0000 UTC m=+237.389598465 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs") pod "router-default-799b44d47c-5c54q" (UID: "09f9b961-e743-4da0-be0d-80a58f646b62") : secret "router-metrics-certs-default" not found
Jul 24 03:21:20.465793 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:20.465751    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-m7lml" event={"ID":"c732f138-0f8d-4ab9-b0b8-a2581b916fa9","Type":"ContainerStarted","Data":"8022e9af167a810dcfc621ee4682932c4493bdc1425751f1e31839cf5a4cc26c"}
Jul 24 03:21:20.527042 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:20.526744    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mb4q7\" (UID: \"697e431d-f03f-4ddb-b7a5-fadefc9b034f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"
Jul 24 03:21:20.527042 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:20.526865    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:20.527042 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:20.526910    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 03:21:20.527042 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:20.526982    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 03:21:20.527042 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:20.526989    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls podName:697e431d-f03f-4ddb-b7a5-fadefc9b034f nodeName:}" failed. No retries permitted until 2026-07-24 03:21:22.526970058 +0000 UTC m=+237.490063325 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls") pod "cluster-samples-operator-6748489656-mb4q7" (UID: "697e431d-f03f-4ddb-b7a5-fadefc9b034f") : secret "samples-operator-tls" not found
Jul 24 03:21:20.527042 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:20.527016    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls podName:136cde74-b9d8-4c3c-9bca-74b7628db11b nodeName:}" failed. No retries permitted until 2026-07-24 03:21:22.527006351 +0000 UTC m=+237.490099616 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-p5ghk" (UID: "136cde74-b9d8-4c3c-9bca-74b7628db11b") : secret "cluster-monitoring-operator-tls" not found
Jul 24 03:21:22.444674 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.444633    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:22.445144 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.444741    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:22.445144 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:22.444829    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 03:21:22.445144 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:22.444915    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs podName:09f9b961-e743-4da0-be0d-80a58f646b62 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:26.444893709 +0000 UTC m=+241.407986994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs") pod "router-default-799b44d47c-5c54q" (UID: "09f9b961-e743-4da0-be0d-80a58f646b62") : secret "router-metrics-certs-default" not found
Jul 24 03:21:22.445144 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:22.444932    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle podName:09f9b961-e743-4da0-be0d-80a58f646b62 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:26.444926693 +0000 UTC m=+241.408019958 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle") pod "router-default-799b44d47c-5c54q" (UID: "09f9b961-e743-4da0-be0d-80a58f646b62") : configmap references non-existent config key: service-ca.crt
Jul 24 03:21:22.473059 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.473032    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/0.log"
Jul 24 03:21:22.473236 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.473074    2574 generic.go:356] "Generic (PLEG): container finished" podID="69510a47-b326-4920-acb8-1baa907c926d" containerID="76b22e1ff428cf0bb4bce673250117e97b86efc9f282035374622a53b2874042" exitCode=255
Jul 24 03:21:22.473236 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.473151    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp" event={"ID":"69510a47-b326-4920-acb8-1baa907c926d","Type":"ContainerDied","Data":"76b22e1ff428cf0bb4bce673250117e97b86efc9f282035374622a53b2874042"}
Jul 24 03:21:22.473473 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.473451    2574 scope.go:117] "RemoveContainer" containerID="76b22e1ff428cf0bb4bce673250117e97b86efc9f282035374622a53b2874042"
Jul 24 03:21:22.474908 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.474884    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xbwck" event={"ID":"3af1a82c-c889-4da3-95fa-6812a25bcea3","Type":"ContainerStarted","Data":"426da881e37fcfc13ef8402c5555eac77ac9ce3958d107a139f1796fea4e193e"}
Jul 24 03:21:22.476502 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.476482    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp" event={"ID":"bc7cb98c-bd37-44bf-80f0-1cd615e8a36f","Type":"ContainerStarted","Data":"d85cfcda00386fb90c4670651236c953d7615aae1f82b83fc7a05f016089547a"}
Jul 24 03:21:22.489409 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.489357    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-m7lml" podStartSLOduration=4.489334979 podStartE2EDuration="4.489334979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:21:20.479489477 +0000 UTC m=+235.442582768" watchObservedRunningTime="2026-07-24 03:21:22.489334979 +0000 UTC m=+237.452428267"
Jul 24 03:21:22.502349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.502297    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp" podStartSLOduration=1.915443899 podStartE2EDuration="4.502282674s" totalImagesPullingTime="2.586838775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:21:22.501656409 +0000 UTC m=+237.464749695" watchObservedRunningTime="2026-07-24 03:21:22.502282674 +0000 UTC m=+237.465375964"
Jul 24 03:21:22.517031 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.516968    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-xbwck" podStartSLOduration=2.168150736 podStartE2EDuration="4.516943393s" totalImagesPullingTime="2.348792657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:21:22.515174202 +0000 UTC m=+237.478267490" watchObservedRunningTime="2026-07-24 03:21:22.516943393 +0000 UTC m=+237.480036684"
Jul 24 03:21:22.546034 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.546000    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:22.546187 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.546089    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mb4q7\" (UID: \"697e431d-f03f-4ddb-b7a5-fadefc9b034f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"
Jul 24 03:21:22.546187 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:22.546173    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 03:21:22.546298 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:22.546223    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 03:21:22.546298 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:22.546235    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls podName:136cde74-b9d8-4c3c-9bca-74b7628db11b nodeName:}" failed. No retries permitted until 2026-07-24 03:21:26.546220491 +0000 UTC m=+241.509313756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-p5ghk" (UID: "136cde74-b9d8-4c3c-9bca-74b7628db11b") : secret "cluster-monitoring-operator-tls" not found
Jul 24 03:21:22.546298 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:22.546272    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls podName:697e431d-f03f-4ddb-b7a5-fadefc9b034f nodeName:}" failed. No retries permitted until 2026-07-24 03:21:26.54625693 +0000 UTC m=+241.509350208 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls") pod "cluster-samples-operator-6748489656-mb4q7" (UID: "697e431d-f03f-4ddb-b7a5-fadefc9b034f") : secret "samples-operator-tls" not found
Jul 24 03:21:22.964211 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.962023    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x95zd"]
Jul 24 03:21:22.966762 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.966736    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:22.969286 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.969255    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 03:21:22.969483 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.969258    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fs6xn\""
Jul 24 03:21:22.969483 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.969266    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 03:21:22.969483 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.969406    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 03:21:22.969948 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.969933    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 03:21:22.973571 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:22.973552    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x95zd"]
Jul 24 03:21:23.051129 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.051090    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bef52dbb-50ae-4945-8232-4da340d6439d-data-volume\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.051316 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.051175    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zw4s\" (UniqueName: \"kubernetes.io/projected/bef52dbb-50ae-4945-8232-4da340d6439d-kube-api-access-6zw4s\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.051316 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.051195    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bef52dbb-50ae-4945-8232-4da340d6439d-crio-socket\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.051316 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.051224    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.051316 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.051274    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.152609 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.152548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.152821 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.152636    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bef52dbb-50ae-4945-8232-4da340d6439d-data-volume\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.152821 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:23.152709    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:23.152821 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.152734    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zw4s\" (UniqueName: \"kubernetes.io/projected/bef52dbb-50ae-4945-8232-4da340d6439d-kube-api-access-6zw4s\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.152821 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.152760    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bef52dbb-50ae-4945-8232-4da340d6439d-crio-socket\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.152821 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:23.152785    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls podName:bef52dbb-50ae-4945-8232-4da340d6439d nodeName:}" failed. No retries permitted until 2026-07-24 03:21:23.652769007 +0000 UTC m=+238.615862271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x95zd" (UID: "bef52dbb-50ae-4945-8232-4da340d6439d") : secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:23.152821 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.152814    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.153090 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:23.152870    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 03:21:23.153090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.152818    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bef52dbb-50ae-4945-8232-4da340d6439d-crio-socket\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.153090 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:23.152946    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm podName:bef52dbb-50ae-4945-8232-4da340d6439d nodeName:}" failed. No retries permitted until 2026-07-24 03:21:23.652928106 +0000 UTC m=+238.616021375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x95zd" (UID: "bef52dbb-50ae-4945-8232-4da340d6439d") : configmap "kube-rbac-proxy" not found
Jul 24 03:21:23.153090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.152985    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bef52dbb-50ae-4945-8232-4da340d6439d-data-volume\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.160730 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.160701    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zw4s\" (UniqueName: \"kubernetes.io/projected/bef52dbb-50ae-4945-8232-4da340d6439d-kube-api-access-6zw4s\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.481471 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.481414    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:21:23.481911 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.481891    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/0.log"
Jul 24 03:21:23.481952 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.481926    2574 generic.go:356] "Generic (PLEG): container finished" podID="69510a47-b326-4920-acb8-1baa907c926d" containerID="2e38883b9ed8258b55bb5ecd5ef9cde17846e48c20284eb7226ef0b8546c8c17" exitCode=255
Jul 24 03:21:23.482038 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.482016    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp" event={"ID":"69510a47-b326-4920-acb8-1baa907c926d","Type":"ContainerDied","Data":"2e38883b9ed8258b55bb5ecd5ef9cde17846e48c20284eb7226ef0b8546c8c17"}
Jul 24 03:21:23.482088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.482077    2574 scope.go:117] "RemoveContainer" containerID="76b22e1ff428cf0bb4bce673250117e97b86efc9f282035374622a53b2874042"
Jul 24 03:21:23.482371 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.482352    2574 scope.go:117] "RemoveContainer" containerID="2e38883b9ed8258b55bb5ecd5ef9cde17846e48c20284eb7226ef0b8546c8c17"
Jul 24 03:21:23.482598 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:23.482577    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-bjpkp_openshift-console-operator(69510a47-b326-4920-acb8-1baa907c926d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp" podUID="69510a47-b326-4920-acb8-1baa907c926d"
Jul 24 03:21:23.657264 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.657207    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.657492 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:23.657330    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:23.657492 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:23.657452    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls podName:bef52dbb-50ae-4945-8232-4da340d6439d nodeName:}" failed. No retries permitted until 2026-07-24 03:21:24.657404711 +0000 UTC m=+239.620497995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x95zd" (UID: "bef52dbb-50ae-4945-8232-4da340d6439d") : secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:23.657841 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:23.657815    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:23.657928 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:23.657917    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 03:21:23.657981 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:23.657970    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm podName:bef52dbb-50ae-4945-8232-4da340d6439d nodeName:}" failed. No retries permitted until 2026-07-24 03:21:24.657955006 +0000 UTC m=+239.621048285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x95zd" (UID: "bef52dbb-50ae-4945-8232-4da340d6439d") : configmap "kube-rbac-proxy" not found
Jul 24 03:21:24.486730 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:24.486701    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:21:24.487204 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:24.487030    2574 scope.go:117] "RemoveContainer" containerID="2e38883b9ed8258b55bb5ecd5ef9cde17846e48c20284eb7226ef0b8546c8c17"
Jul 24 03:21:24.487204 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:24.487172    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-bjpkp_openshift-console-operator(69510a47-b326-4920-acb8-1baa907c926d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp" podUID="69510a47-b326-4920-acb8-1baa907c926d"
Jul 24 03:21:24.665326 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:24.665284    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:24.665546 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:24.665366    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 03:21:24.665546 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:24.665463    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm podName:bef52dbb-50ae-4945-8232-4da340d6439d nodeName:}" failed. No retries permitted until 2026-07-24 03:21:26.665421746 +0000 UTC m=+241.628515027 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x95zd" (UID: "bef52dbb-50ae-4945-8232-4da340d6439d") : configmap "kube-rbac-proxy" not found
Jul 24 03:21:24.665546 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:24.665532    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:24.665719 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:24.665689    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:24.665785 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:24.665729    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls podName:bef52dbb-50ae-4945-8232-4da340d6439d nodeName:}" failed. No retries permitted until 2026-07-24 03:21:26.665716564 +0000 UTC m=+241.628809829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x95zd" (UID: "bef52dbb-50ae-4945-8232-4da340d6439d") : secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:25.764088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:25.764061    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s7jjg_559deb09-e606-40bb-ba17-b957dca7b41a/dns-node-resolver/0.log"
Jul 24 03:21:26.480130 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:26.480094    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:26.480348 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:26.480170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:26.480348 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:26.480292    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 03:21:26.480348 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:26.480293    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle podName:09f9b961-e743-4da0-be0d-80a58f646b62 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:34.480268534 +0000 UTC m=+249.443361813 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle") pod "router-default-799b44d47c-5c54q" (UID: "09f9b961-e743-4da0-be0d-80a58f646b62") : configmap references non-existent config key: service-ca.crt
Jul 24 03:21:26.480348 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:26.480345    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs podName:09f9b961-e743-4da0-be0d-80a58f646b62 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:34.480332315 +0000 UTC m=+249.443425580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs") pod "router-default-799b44d47c-5c54q" (UID: "09f9b961-e743-4da0-be0d-80a58f646b62") : secret "router-metrics-certs-default" not found
Jul 24 03:21:26.560727 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:26.560698    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5j4wp_4b97304d-62fa-4447-81d8-525c8b6670b3/node-ca/0.log"
Jul 24 03:21:26.580703 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:26.580669    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:26.580811 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:26.580731    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mb4q7\" (UID: \"697e431d-f03f-4ddb-b7a5-fadefc9b034f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"
Jul 24 03:21:26.580849 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:26.580821    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 24 03:21:26.580883 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:26.580870    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 24 03:21:26.580916 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:26.580898    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls podName:136cde74-b9d8-4c3c-9bca-74b7628db11b nodeName:}" failed. No retries permitted until 2026-07-24 03:21:34.580874632 +0000 UTC m=+249.543967913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-p5ghk" (UID: "136cde74-b9d8-4c3c-9bca-74b7628db11b") : secret "cluster-monitoring-operator-tls" not found
Jul 24 03:21:26.580960 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:26.580917    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls podName:697e431d-f03f-4ddb-b7a5-fadefc9b034f nodeName:}" failed. No retries permitted until 2026-07-24 03:21:34.580908192 +0000 UTC m=+249.544001459 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls") pod "cluster-samples-operator-6748489656-mb4q7" (UID: "697e431d-f03f-4ddb-b7a5-fadefc9b034f") : secret "samples-operator-tls" not found
Jul 24 03:21:26.682051 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:26.682008    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:26.682227 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:26.682114    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 03:21:26.682227 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:26.682126    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:26.682227 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:26.682227    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm podName:bef52dbb-50ae-4945-8232-4da340d6439d nodeName:}" failed. No retries permitted until 2026-07-24 03:21:30.682208531 +0000 UTC m=+245.645301796 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x95zd" (UID: "bef52dbb-50ae-4945-8232-4da340d6439d") : configmap "kube-rbac-proxy" not found
Jul 24 03:21:26.682344 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:26.682235    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:26.682344 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:26.682272    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls podName:bef52dbb-50ae-4945-8232-4da340d6439d nodeName:}" failed. No retries permitted until 2026-07-24 03:21:30.682263251 +0000 UTC m=+245.645356516 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x95zd" (UID: "bef52dbb-50ae-4945-8232-4da340d6439d") : secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:27.359811 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:27.359778    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-757zg_fe775e9b-fb53-4c76-b95e-a43de1163af0/serve-healthcheck-canary/0.log"
Jul 24 03:21:28.360630 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:28.360597    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp_bc7cb98c-bd37-44bf-80f0-1cd615e8a36f/kube-storage-version-migrator-operator/0.log"
Jul 24 03:21:29.047039 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:29.046949    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:29.047039 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:29.047012    2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:29.047362 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:29.047349    2574 scope.go:117] "RemoveContainer" containerID="2e38883b9ed8258b55bb5ecd5ef9cde17846e48c20284eb7226ef0b8546c8c17"
Jul 24 03:21:29.047536 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:29.047519    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-bjpkp_openshift-console-operator(69510a47-b326-4920-acb8-1baa907c926d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp" podUID="69510a47-b326-4920-acb8-1baa907c926d"
Jul 24 03:21:30.719646 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:30.719606    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:30.720141 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:30.719677    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:30.720141 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:30.719760    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 03:21:30.720141 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:30.719790    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:30.720141 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:30.719839    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm podName:bef52dbb-50ae-4945-8232-4da340d6439d nodeName:}" failed. No retries permitted until 2026-07-24 03:21:38.719822925 +0000 UTC m=+253.682916193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x95zd" (UID: "bef52dbb-50ae-4945-8232-4da340d6439d") : configmap "kube-rbac-proxy" not found
Jul 24 03:21:30.720141 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:30.719854    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls podName:bef52dbb-50ae-4945-8232-4da340d6439d nodeName:}" failed. No retries permitted until 2026-07-24 03:21:38.719848316 +0000 UTC m=+253.682941580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x95zd" (UID: "bef52dbb-50ae-4945-8232-4da340d6439d") : secret "insights-runtime-extractor-tls" not found
Jul 24 03:21:34.549936 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.549895    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:34.550324 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.549978    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:34.550596 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.550579    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09f9b961-e743-4da0-be0d-80a58f646b62-service-ca-bundle\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:34.552457 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.552439    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/09f9b961-e743-4da0-be0d-80a58f646b62-metrics-certs\") pod \"router-default-799b44d47c-5c54q\" (UID: \"09f9b961-e743-4da0-be0d-80a58f646b62\") " pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:34.650783 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.650735    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:34.650974 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.650809    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mb4q7\" (UID: \"697e431d-f03f-4ddb-b7a5-fadefc9b034f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"
Jul 24 03:21:34.653190 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.653166    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-288bb\""
Jul 24 03:21:34.653400 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.653381    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/136cde74-b9d8-4c3c-9bca-74b7628db11b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-p5ghk\" (UID: \"136cde74-b9d8-4c3c-9bca-74b7628db11b\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:34.653690 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.653669    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/697e431d-f03f-4ddb-b7a5-fadefc9b034f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mb4q7\" (UID: \"697e431d-f03f-4ddb-b7a5-fadefc9b034f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"
Jul 24 03:21:34.662040 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.662018    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:34.726300 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.726266    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-cgfwq\""
Jul 24 03:21:34.732330 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.732303    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2s5n9\""
Jul 24 03:21:34.734014 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.733990    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"
Jul 24 03:21:34.741079 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.741058    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"
Jul 24 03:21:34.811362 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.811288    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-799b44d47c-5c54q"]
Jul 24 03:21:34.814107 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:21:34.813759    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f9b961_e743_4da0_be0d_80a58f646b62.slice/crio-8c78c9228bc99404eb09b9c415355e36e5e038eb49c37ddad1ab0f5ca7ea759b WatchSource:0}: Error finding container 8c78c9228bc99404eb09b9c415355e36e5e038eb49c37ddad1ab0f5ca7ea759b: Status 404 returned error can't find the container with id 8c78c9228bc99404eb09b9c415355e36e5e038eb49c37ddad1ab0f5ca7ea759b
Jul 24 03:21:34.886960 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.886929    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7"]
Jul 24 03:21:34.903467 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:34.903034    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk"]
Jul 24 03:21:34.907060 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:21:34.907030    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136cde74_b9d8_4c3c_9bca_74b7628db11b.slice/crio-66f7c859ca864d30cbee8541d3dd49d876af4f7e979f8be776973b0616123e44 WatchSource:0}: Error finding container 66f7c859ca864d30cbee8541d3dd49d876af4f7e979f8be776973b0616123e44: Status 404 returned error can't find the container with id 66f7c859ca864d30cbee8541d3dd49d876af4f7e979f8be776973b0616123e44
Jul 24 03:21:35.521949 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:35.521903    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7" event={"ID":"697e431d-f03f-4ddb-b7a5-fadefc9b034f","Type":"ContainerStarted","Data":"a10c4536968edf8613a46d373e76f04313d3ba5a4c2fc524a7d1f00e39d27476"}
Jul 24 03:21:35.523558 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:35.523525    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-799b44d47c-5c54q" event={"ID":"09f9b961-e743-4da0-be0d-80a58f646b62","Type":"ContainerStarted","Data":"0ec7441f94590e253dcae4496fa183cc25cd226e26c96d86070a2226dbf694c3"}
Jul 24 03:21:35.523728 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:35.523577    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-799b44d47c-5c54q" event={"ID":"09f9b961-e743-4da0-be0d-80a58f646b62","Type":"ContainerStarted","Data":"8c78c9228bc99404eb09b9c415355e36e5e038eb49c37ddad1ab0f5ca7ea759b"}
Jul 24 03:21:35.524877 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:35.524850    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk" event={"ID":"136cde74-b9d8-4c3c-9bca-74b7628db11b","Type":"ContainerStarted","Data":"66f7c859ca864d30cbee8541d3dd49d876af4f7e979f8be776973b0616123e44"}
Jul 24 03:21:35.541639 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:35.541550    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-799b44d47c-5c54q" podStartSLOduration=17.541530655 podStartE2EDuration="17.541530655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:21:35.541443641 +0000 UTC m=+250.504536919" watchObservedRunningTime="2026-07-24 03:21:35.541530655 +0000 UTC m=+250.504623942"
Jul 24 03:21:35.662599 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:35.662564    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:35.665466 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:35.665416    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:36.364247 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:36.364212    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:21:36.367159 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:36.367130    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d26b109-0e00-4704-95a2-9cda7952facb-metrics-certs\") pod \"network-metrics-daemon-mp65w\" (UID: \"6d26b109-0e00-4704-95a2-9cda7952facb\") " pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:21:36.528283 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:36.528248    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:36.529804 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:36.529779    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-799b44d47c-5c54q"
Jul 24 03:21:36.570249 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:36.570198    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-frl8x\""
Jul 24 03:21:36.579122 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:36.579089    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mp65w"
Jul 24 03:21:37.194448 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:37.194360    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mp65w"]
Jul 24 03:21:37.200627 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:21:37.200593    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d26b109_0e00_4704_95a2_9cda7952facb.slice/crio-7176c4b8d81de057467235b25e87889a9f24e02d6c163f344276482cb9a17679 WatchSource:0}: Error finding container 7176c4b8d81de057467235b25e87889a9f24e02d6c163f344276482cb9a17679: Status 404 returned error can't find the container with id 7176c4b8d81de057467235b25e87889a9f24e02d6c163f344276482cb9a17679
Jul 24 03:21:37.532712 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:37.532679    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk" event={"ID":"136cde74-b9d8-4c3c-9bca-74b7628db11b","Type":"ContainerStarted","Data":"2e6f05365135d3a3c9f8ba203cc11cb8af01784c19f91a0a5e22f8ef0284e03c"}
Jul 24 03:21:37.534001 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:37.533974    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mp65w" event={"ID":"6d26b109-0e00-4704-95a2-9cda7952facb","Type":"ContainerStarted","Data":"7176c4b8d81de057467235b25e87889a9f24e02d6c163f344276482cb9a17679"}
Jul 24 03:21:37.535804 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:37.535779    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7" event={"ID":"697e431d-f03f-4ddb-b7a5-fadefc9b034f","Type":"ContainerStarted","Data":"c0eb514981085552b7918b8696c41ce9662db18a62ba1801998a274e9edd1d62"}
Jul 24 03:21:37.535902 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:37.535822    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7" event={"ID":"697e431d-f03f-4ddb-b7a5-fadefc9b034f","Type":"ContainerStarted","Data":"4aae701d80ad31c3b13bdea8dc27c0b0a3b309b67f38eeeee2aeb516c71c3a4d"}
Jul 24 03:21:37.562837 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:37.562775    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-p5ghk" podStartSLOduration=17.412394503 podStartE2EDuration="19.562758196s" totalImagesPullingTime="2.150363693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:21:37.562101302 +0000 UTC m=+252.525194588" watchObservedRunningTime="2026-07-24 03:21:37.562758196 +0000 UTC m=+252.525851477"
Jul 24 03:21:37.587659 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:37.587604    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mb4q7" podStartSLOduration=17.466436349 podStartE2EDuration="19.587590555s" totalImagesPullingTime="2.121154206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:21:37.586769509 +0000 UTC m=+252.549862836" watchObservedRunningTime="2026-07-24 03:21:37.587590555 +0000 UTC m=+252.550683841"
Jul 24 03:21:38.541675 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:38.541576    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mp65w" event={"ID":"6d26b109-0e00-4704-95a2-9cda7952facb","Type":"ContainerStarted","Data":"50284d5157f15963f644254bc8613adce37e1cf602a11debfa2f636e5fd636ed"}
Jul 24 03:21:38.542049 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:38.541694    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mp65w" event={"ID":"6d26b109-0e00-4704-95a2-9cda7952facb","Type":"ContainerStarted","Data":"644baae28b00de94f68c2aa5d60bae5a5e0d44546df95be9a9966c4015070a13"}
Jul 24 03:21:38.557212 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:38.557157    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mp65w" podStartSLOduration=251.467870648 podStartE2EDuration="4m12.557140453s" totalImagesPullingTime="1.089269805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:21:38.555968261 +0000 UTC m=+253.519061561" watchObservedRunningTime="2026-07-24 03:21:38.557140453 +0000 UTC m=+253.520233798"
Jul 24 03:21:38.786249 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:38.786191    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:38.786463 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:38.786295    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:38.786762 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:38.786741    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bef52dbb-50ae-4945-8232-4da340d6439d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:38.788784 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:38.788762    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bef52dbb-50ae-4945-8232-4da340d6439d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x95zd\" (UID: \"bef52dbb-50ae-4945-8232-4da340d6439d\") " pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:38.879419 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:38.879337    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fs6xn\""
Jul 24 03:21:38.887349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:38.887325    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x95zd"
Jul 24 03:21:39.018320 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:39.018292    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x95zd"]
Jul 24 03:21:39.020669 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:21:39.020646    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbef52dbb_50ae_4945_8232_4da340d6439d.slice/crio-11bb97a46b3e6976d3a9eb82e4cdf2ee1e2eda34f2eab5d4ea33697f24003e02 WatchSource:0}: Error finding container 11bb97a46b3e6976d3a9eb82e4cdf2ee1e2eda34f2eab5d4ea33697f24003e02: Status 404 returned error can't find the container with id 11bb97a46b3e6976d3a9eb82e4cdf2ee1e2eda34f2eab5d4ea33697f24003e02
Jul 24 03:21:39.548805 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:39.548769    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x95zd" event={"ID":"bef52dbb-50ae-4945-8232-4da340d6439d","Type":"ContainerStarted","Data":"11bb97a46b3e6976d3a9eb82e4cdf2ee1e2eda34f2eab5d4ea33697f24003e02"}
Jul 24 03:21:42.567807 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:42.567774    2574 scope.go:117] "RemoveContainer" containerID="2e38883b9ed8258b55bb5ecd5ef9cde17846e48c20284eb7226ef0b8546c8c17"
Jul 24 03:21:44.572137 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:44.571896    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:21:44.572137 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:44.572006    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp" event={"ID":"69510a47-b326-4920-acb8-1baa907c926d","Type":"ContainerStarted","Data":"686a4d23e51157f9804b62011c65d78698be53ee6ce990b4845f0def88d516cc"}
Jul 24 03:21:44.572668 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:44.572532    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:44.574831 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:44.574806    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x95zd" event={"ID":"bef52dbb-50ae-4945-8232-4da340d6439d","Type":"ContainerStarted","Data":"db8afb665a653ef63eed7198052c49cabbdc77fc2b9e491b313c2b4af4585b1a"}
Jul 24 03:21:45.076928 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.076893    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp"
Jul 24 03:21:45.094325 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.094261    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-bjpkp" podStartSLOduration=24.562152903 podStartE2EDuration="27.094237932s" totalImagesPullingTime="2.532085029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:21:44.590952557 +0000 UTC m=+259.554045842" watchObservedRunningTime="2026-07-24 03:21:45.094237932 +0000 UTC m=+260.057331221"
Jul 24 03:21:45.094617 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.094590    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74ffdd85b9-6cwrn"]
Jul 24 03:21:45.094864 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:45.094839    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn" podUID="8d3b50f3-2285-4c78-9972-4964483c7837"
Jul 24 03:21:45.349284 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.349189    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-48xzj"]
Jul 24 03:21:45.353263 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.353199    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-48xzj"
Jul 24 03:21:45.356072 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.356041    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 24 03:21:45.356228 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.356056    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 24 03:21:45.356991 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.356498    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-w4hrg\""
Jul 24 03:21:45.368613 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.368581    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-48xzj"]
Jul 24 03:21:45.445900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.445867    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph47z\" (UniqueName: \"kubernetes.io/projected/985df3af-4c44-462b-90e2-46bbe9a6b181-kube-api-access-ph47z\") pod \"downloads-7c97dc4555-48xzj\" (UID: \"985df3af-4c44-462b-90e2-46bbe9a6b181\") " pod="openshift-console/downloads-7c97dc4555-48xzj"
Jul 24 03:21:45.546995 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.546945    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph47z\" (UniqueName: \"kubernetes.io/projected/985df3af-4c44-462b-90e2-46bbe9a6b181-kube-api-access-ph47z\") pod \"downloads-7c97dc4555-48xzj\" (UID: \"985df3af-4c44-462b-90e2-46bbe9a6b181\") " pod="openshift-console/downloads-7c97dc4555-48xzj"
Jul 24 03:21:45.559812 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.559786    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph47z\" (UniqueName: \"kubernetes.io/projected/985df3af-4c44-462b-90e2-46bbe9a6b181-kube-api-access-ph47z\") pod \"downloads-7c97dc4555-48xzj\" (UID: \"985df3af-4c44-462b-90e2-46bbe9a6b181\") " pod="openshift-console/downloads-7c97dc4555-48xzj"
Jul 24 03:21:45.577736 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.577708    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:21:45.584775 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.584752    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:21:45.648114 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.648078    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d3b50f3-2285-4c78-9972-4964483c7837-ca-trust-extracted\") pod \"8d3b50f3-2285-4c78-9972-4964483c7837\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") "
Jul 24 03:21:45.648308 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.648130    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d3b50f3-2285-4c78-9972-4964483c7837-installation-pull-secrets\") pod \"8d3b50f3-2285-4c78-9972-4964483c7837\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") "
Jul 24 03:21:45.648308 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.648160    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qf9sz\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-kube-api-access-qf9sz\") pod \"8d3b50f3-2285-4c78-9972-4964483c7837\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") "
Jul 24 03:21:45.648308 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.648179    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d3b50f3-2285-4c78-9972-4964483c7837-trusted-ca\") pod \"8d3b50f3-2285-4c78-9972-4964483c7837\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") "
Jul 24 03:21:45.648308 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.648217    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d3b50f3-2285-4c78-9972-4964483c7837-registry-certificates\") pod \"8d3b50f3-2285-4c78-9972-4964483c7837\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") "
Jul 24 03:21:45.648308 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.648237    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d3b50f3-2285-4c78-9972-4964483c7837-image-registry-private-configuration\") pod \"8d3b50f3-2285-4c78-9972-4964483c7837\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") "
Jul 24 03:21:45.648308 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.648274    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-bound-sa-token\") pod \"8d3b50f3-2285-4c78-9972-4964483c7837\" (UID: \"8d3b50f3-2285-4c78-9972-4964483c7837\") "
Jul 24 03:21:45.648708 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.648414    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d3b50f3-2285-4c78-9972-4964483c7837-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8d3b50f3-2285-4c78-9972-4964483c7837" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:21:45.648708 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.648538    2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d3b50f3-2285-4c78-9972-4964483c7837-ca-trust-extracted\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:21:45.648812 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.648776    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d3b50f3-2285-4c78-9972-4964483c7837-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8d3b50f3-2285-4c78-9972-4964483c7837" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:21:45.649053 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.648985    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d3b50f3-2285-4c78-9972-4964483c7837-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8d3b50f3-2285-4c78-9972-4964483c7837" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:21:45.650855 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.650831    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d3b50f3-2285-4c78-9972-4964483c7837-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8d3b50f3-2285-4c78-9972-4964483c7837" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:21:45.650953 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.650890    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-kube-api-access-qf9sz" (OuterVolumeSpecName: "kube-api-access-qf9sz") pod "8d3b50f3-2285-4c78-9972-4964483c7837" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837"). InnerVolumeSpecName "kube-api-access-qf9sz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:21:45.650953 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.650925    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8d3b50f3-2285-4c78-9972-4964483c7837" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:21:45.651289 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.651266    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d3b50f3-2285-4c78-9972-4964483c7837-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8d3b50f3-2285-4c78-9972-4964483c7837" (UID: "8d3b50f3-2285-4c78-9972-4964483c7837"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:21:45.667232 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.667029    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-48xzj"
Jul 24 03:21:45.749745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.749711    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qf9sz\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-kube-api-access-qf9sz\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:21:45.749745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.749742    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d3b50f3-2285-4c78-9972-4964483c7837-trusted-ca\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:21:45.749745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.749752    2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d3b50f3-2285-4c78-9972-4964483c7837-registry-certificates\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:21:45.749985 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.749761    2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d3b50f3-2285-4c78-9972-4964483c7837-image-registry-private-configuration\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:21:45.749985 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.749771    2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-bound-sa-token\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:21:45.749985 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:45.749779    2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d3b50f3-2285-4c78-9972-4964483c7837-installation-pull-secrets\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:21:46.581938 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:46.581905    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74ffdd85b9-6cwrn"
Jul 24 03:21:46.613788 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:46.613758    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74ffdd85b9-6cwrn"]
Jul 24 03:21:46.617518 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:46.617488    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-74ffdd85b9-6cwrn"]
Jul 24 03:21:46.656652 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:46.656623    2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d3b50f3-2285-4c78-9972-4964483c7837-registry-tls\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:21:47.571821 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:47.571788    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d3b50f3-2285-4c78-9972-4964483c7837" path="/var/lib/kubelet/pods/8d3b50f3-2285-4c78-9972-4964483c7837/volumes"
Jul 24 03:21:47.688702 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:47.688672    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-48xzj"]
Jul 24 03:21:47.691365 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:21:47.691334    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985df3af_4c44_462b_90e2_46bbe9a6b181.slice/crio-5963793da23d29bda5fae289b050f804709a0f8dd5b5c49f5dfd527e6f51542d WatchSource:0}: Error finding container 5963793da23d29bda5fae289b050f804709a0f8dd5b5c49f5dfd527e6f51542d: Status 404 returned error can't find the container with id 5963793da23d29bda5fae289b050f804709a0f8dd5b5c49f5dfd527e6f51542d
Jul 24 03:21:48.592980 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:48.592905    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-48xzj" event={"ID":"985df3af-4c44-462b-90e2-46bbe9a6b181","Type":"ContainerStarted","Data":"5963793da23d29bda5fae289b050f804709a0f8dd5b5c49f5dfd527e6f51542d"}
Jul 24 03:21:48.596882 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:48.596846    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x95zd" event={"ID":"bef52dbb-50ae-4945-8232-4da340d6439d","Type":"ContainerStarted","Data":"5369a13c31b88e7eefdda414107e9cbd2c84aa9e6dc40ea19f89b2b220d3dd84"}
Jul 24 03:21:50.638915 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.638875    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-465j4"]
Jul 24 03:21:50.653915 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.653838    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-465j4"]
Jul 24 03:21:50.654083 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.653966    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:50.656288 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.656263    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 24 03:21:50.656569 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.656418    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-89rhv\""
Jul 24 03:21:50.656569 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.656417    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 24 03:21:50.657103 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.657082    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 03:21:50.798022 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.797977    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3acf96c6-8064-4d64-b8c5-c479d5ecc856-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-465j4\" (UID: \"3acf96c6-8064-4d64-b8c5-c479d5ecc856\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:50.798220 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.798038    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k47fk\" (UniqueName: \"kubernetes.io/projected/3acf96c6-8064-4d64-b8c5-c479d5ecc856-kube-api-access-k47fk\") pod \"prometheus-operator-84d69f794d-465j4\" (UID: \"3acf96c6-8064-4d64-b8c5-c479d5ecc856\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:50.798220 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.798075    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3acf96c6-8064-4d64-b8c5-c479d5ecc856-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-465j4\" (UID: \"3acf96c6-8064-4d64-b8c5-c479d5ecc856\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:50.798220 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.798178    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3acf96c6-8064-4d64-b8c5-c479d5ecc856-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-465j4\" (UID: \"3acf96c6-8064-4d64-b8c5-c479d5ecc856\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:50.899470 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.899366    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3acf96c6-8064-4d64-b8c5-c479d5ecc856-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-465j4\" (UID: \"3acf96c6-8064-4d64-b8c5-c479d5ecc856\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:50.899722 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.899625    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3acf96c6-8064-4d64-b8c5-c479d5ecc856-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-465j4\" (UID: \"3acf96c6-8064-4d64-b8c5-c479d5ecc856\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:50.899813 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.899748    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k47fk\" (UniqueName: \"kubernetes.io/projected/3acf96c6-8064-4d64-b8c5-c479d5ecc856-kube-api-access-k47fk\") pod \"prometheus-operator-84d69f794d-465j4\" (UID: \"3acf96c6-8064-4d64-b8c5-c479d5ecc856\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:50.899813 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.899795    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3acf96c6-8064-4d64-b8c5-c479d5ecc856-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-465j4\" (UID: \"3acf96c6-8064-4d64-b8c5-c479d5ecc856\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:50.900544 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.900188    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3acf96c6-8064-4d64-b8c5-c479d5ecc856-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-465j4\" (UID: \"3acf96c6-8064-4d64-b8c5-c479d5ecc856\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:50.900544 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:50.900305    2574 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 24 03:21:50.900544 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:50.900380    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3acf96c6-8064-4d64-b8c5-c479d5ecc856-prometheus-operator-tls podName:3acf96c6-8064-4d64-b8c5-c479d5ecc856 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:51.400360712 +0000 UTC m=+266.363453977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3acf96c6-8064-4d64-b8c5-c479d5ecc856-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-465j4" (UID: "3acf96c6-8064-4d64-b8c5-c479d5ecc856") : secret "prometheus-operator-tls" not found
Jul 24 03:21:50.903512 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.903482    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3acf96c6-8064-4d64-b8c5-c479d5ecc856-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-465j4\" (UID: \"3acf96c6-8064-4d64-b8c5-c479d5ecc856\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:50.909571 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:50.909540    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k47fk\" (UniqueName: \"kubernetes.io/projected/3acf96c6-8064-4d64-b8c5-c479d5ecc856-kube-api-access-k47fk\") pod \"prometheus-operator-84d69f794d-465j4\" (UID: \"3acf96c6-8064-4d64-b8c5-c479d5ecc856\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:51.404756 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:51.404718    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3acf96c6-8064-4d64-b8c5-c479d5ecc856-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-465j4\" (UID: \"3acf96c6-8064-4d64-b8c5-c479d5ecc856\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:51.407904 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:51.407874    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3acf96c6-8064-4d64-b8c5-c479d5ecc856-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-465j4\" (UID: \"3acf96c6-8064-4d64-b8c5-c479d5ecc856\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:51.566123 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:51.566088    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4"
Jul 24 03:21:51.786356 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:51.786326    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-465j4"]
Jul 24 03:21:51.789312 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:21:51.789283    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3acf96c6_8064_4d64_b8c5_c479d5ecc856.slice/crio-aea439f7f7211c969d8e904223c01b70ac051da9371f75529c1df5c8fb846e61 WatchSource:0}: Error finding container aea439f7f7211c969d8e904223c01b70ac051da9371f75529c1df5c8fb846e61: Status 404 returned error can't find the container with id aea439f7f7211c969d8e904223c01b70ac051da9371f75529c1df5c8fb846e61
Jul 24 03:21:52.615540 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:52.615496    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x95zd" event={"ID":"bef52dbb-50ae-4945-8232-4da340d6439d","Type":"ContainerStarted","Data":"bc5c8201beb4c79d19e4fd14c300b6bb739433c69ada9d9ad8842d076afbf2aa"}
Jul 24 03:21:52.616803 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:52.616738    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4" event={"ID":"3acf96c6-8064-4d64-b8c5-c479d5ecc856","Type":"ContainerStarted","Data":"aea439f7f7211c969d8e904223c01b70ac051da9371f75529c1df5c8fb846e61"}
Jul 24 03:21:52.632354 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:52.632296    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-x95zd" podStartSLOduration=18.083908401 podStartE2EDuration="30.632279638s" totalImagesPullingTime="12.548371237s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 03:21:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:21:52.630991431 +0000 UTC m=+267.594084719" watchObservedRunningTime="2026-07-24 03:21:52.632279638 +0000 UTC m=+267.595372924"
Jul 24 03:21:53.624341 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:53.624255    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4" event={"ID":"3acf96c6-8064-4d64-b8c5-c479d5ecc856","Type":"ContainerStarted","Data":"0ee2dd90af82c9036895531399e7fd8b1b9a8399ab3a2585ac5581aeb56ce9ab"}
Jul 24 03:21:53.624341 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:53.624316    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4" event={"ID":"3acf96c6-8064-4d64-b8c5-c479d5ecc856","Type":"ContainerStarted","Data":"aeb36b19dd4a23bde981d6090c16a3791b7c72644e9ed91673510d10832778bf"}
Jul 24 03:21:53.640854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:53.640797    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-465j4" podStartSLOduration=2.268255455 podStartE2EDuration="3.640703914s" totalImagesPullingTime="1.372448459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:21:53.639030896 +0000 UTC m=+268.602124187" watchObservedRunningTime="2026-07-24 03:21:53.640703914 +0000 UTC m=+268.603797198"
Jul 24 03:21:55.962688 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:55.962652    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"]
Jul 24 03:21:55.966710 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:55.966685    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:55.969911 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:55.969498    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 24 03:21:55.969911 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:55.969507    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-s89zp\""
Jul 24 03:21:55.969911 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:55.969521    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 24 03:21:55.977532 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:55.977460    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"]
Jul 24 03:21:55.985768 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:55.985741    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"]
Jul 24 03:21:55.989112 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:55.989088    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:55.991227 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:55.991208    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 24 03:21:55.991414 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:55.991390    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-pnrmb\""
Jul 24 03:21:55.991610 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:55.991578    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 24 03:21:55.991923 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:55.991898    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 24 03:21:55.999796 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:55.999505    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2vsf6"]
Jul 24 03:21:56.006252 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.005976    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"]
Jul 24 03:21:56.006252 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.006120    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.008768 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.008742    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 03:21:56.008768 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.008763    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2wwgj\""
Jul 24 03:21:56.008927 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.008860    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 03:21:56.008927 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.008764    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 03:21:56.039566 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.039528    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2c90cb8-5136-428b-920b-0db0144e03ba-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-q7nq8\" (UID: \"b2c90cb8-5136-428b-920b-0db0144e03ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:56.039732 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.039614    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2c90cb8-5136-428b-920b-0db0144e03ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-q7nq8\" (UID: \"b2c90cb8-5136-428b-920b-0db0144e03ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:56.039732 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.039645    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgv4r\" (UniqueName: \"kubernetes.io/projected/b2c90cb8-5136-428b-920b-0db0144e03ba-kube-api-access-pgv4r\") pod \"openshift-state-metrics-b8f49f5c6-q7nq8\" (UID: \"b2c90cb8-5136-428b-920b-0db0144e03ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:56.039732 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.039681    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2c90cb8-5136-428b-920b-0db0144e03ba-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-q7nq8\" (UID: \"b2c90cb8-5136-428b-920b-0db0144e03ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:56.140876 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.140836    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2c90cb8-5136-428b-920b-0db0144e03ba-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-q7nq8\" (UID: \"b2c90cb8-5136-428b-920b-0db0144e03ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:56.141053 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.140882    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.141053 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.140993    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/737a1d45-07a7-41b6-9e00-28c489b71173-root\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.141053 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141020    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-wtmp\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.141194 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141063    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2c90cb8-5136-428b-920b-0db0144e03ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-q7nq8\" (UID: \"b2c90cb8-5136-428b-920b-0db0144e03ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:56.141194 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141080    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgv4r\" (UniqueName: \"kubernetes.io/projected/b2c90cb8-5136-428b-920b-0db0144e03ba-kube-api-access-pgv4r\") pod \"openshift-state-metrics-b8f49f5c6-q7nq8\" (UID: \"b2c90cb8-5136-428b-920b-0db0144e03ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:56.141292 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141227    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngkpj\" (UniqueName: \"kubernetes.io/projected/737a1d45-07a7-41b6-9e00-28c489b71173-kube-api-access-ngkpj\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.141335 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141288    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-textfile\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.141387 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141332    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2c90cb8-5136-428b-920b-0db0144e03ba-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-q7nq8\" (UID: \"b2c90cb8-5136-428b-920b-0db0144e03ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:56.141463 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141375    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.141463 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141412    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.141574 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141515    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-accelerators-collector-config\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.141574 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141555    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.141679 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141582    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5v6r\" (UniqueName: \"kubernetes.io/projected/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-api-access-t5v6r\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.141679 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141613    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/737a1d45-07a7-41b6-9e00-28c489b71173-sys\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.141774 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141682    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/654d60b6-080a-4e5f-8a60-544e5986bf00-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.141822 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141805    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/737a1d45-07a7-41b6-9e00-28c489b71173-metrics-client-ca\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.141880 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141839    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-tls\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.141942 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.141877    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654d60b6-080a-4e5f-8a60-544e5986bf00-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.142123 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.142080    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2c90cb8-5136-428b-920b-0db0144e03ba-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-q7nq8\" (UID: \"b2c90cb8-5136-428b-920b-0db0144e03ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:56.144353 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.144329    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2c90cb8-5136-428b-920b-0db0144e03ba-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-q7nq8\" (UID: \"b2c90cb8-5136-428b-920b-0db0144e03ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:56.144490 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.144352    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2c90cb8-5136-428b-920b-0db0144e03ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-q7nq8\" (UID: \"b2c90cb8-5136-428b-920b-0db0144e03ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:56.149869 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.149847    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgv4r\" (UniqueName: \"kubernetes.io/projected/b2c90cb8-5136-428b-920b-0db0144e03ba-kube-api-access-pgv4r\") pod \"openshift-state-metrics-b8f49f5c6-q7nq8\" (UID: \"b2c90cb8-5136-428b-920b-0db0144e03ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:56.242358 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242268    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/737a1d45-07a7-41b6-9e00-28c489b71173-sys\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.242358 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242314    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/654d60b6-080a-4e5f-8a60-544e5986bf00-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.242358 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242332    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/737a1d45-07a7-41b6-9e00-28c489b71173-metrics-client-ca\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.242631 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242410    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/737a1d45-07a7-41b6-9e00-28c489b71173-sys\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.242631 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242504    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-tls\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.242631 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242538    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654d60b6-080a-4e5f-8a60-544e5986bf00-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.242631 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242587    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.242631 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242616    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/737a1d45-07a7-41b6-9e00-28c489b71173-root\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.242797 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242640    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-wtmp\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.242797 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242687    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngkpj\" (UniqueName: \"kubernetes.io/projected/737a1d45-07a7-41b6-9e00-28c489b71173-kube-api-access-ngkpj\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.242797 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242729    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-textfile\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.242797 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242741    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/654d60b6-080a-4e5f-8a60-544e5986bf00-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.242797 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242755    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.242797 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242783    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.243034 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242812    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/737a1d45-07a7-41b6-9e00-28c489b71173-root\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.243034 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242824    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-accelerators-collector-config\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.243034 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242855    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.243034 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242880    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5v6r\" (UniqueName: \"kubernetes.io/projected/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-api-access-t5v6r\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.243034 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:56.242908    2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 24 03:21:56.243034 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.242919    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/737a1d45-07a7-41b6-9e00-28c489b71173-metrics-client-ca\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.243034 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:56.242966    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-tls podName:737a1d45-07a7-41b6-9e00-28c489b71173 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:56.742947168 +0000 UTC m=+271.706040447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-tls") pod "node-exporter-2vsf6" (UID: "737a1d45-07a7-41b6-9e00-28c489b71173") : secret "node-exporter-tls" not found
Jul 24 03:21:56.243034 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:56.242994    2574 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 24 03:21:56.243034 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:21:56.243039    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-state-metrics-tls podName:654d60b6-080a-4e5f-8a60-544e5986bf00 nodeName:}" failed. No retries permitted until 2026-07-24 03:21:56.743028553 +0000 UTC m=+271.706121821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-v9d9r" (UID: "654d60b6-080a-4e5f-8a60-544e5986bf00") : secret "kube-state-metrics-tls" not found
Jul 24 03:21:56.243503 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.243151    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-wtmp\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.243704 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.243682    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-textfile\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.243760 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.243716    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-accelerators-collector-config\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.243760 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.243741    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654d60b6-080a-4e5f-8a60-544e5986bf00-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.244420 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.244335    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.246196 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.246174    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.246490 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.246472    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.250934 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.250914    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngkpj\" (UniqueName: \"kubernetes.io/projected/737a1d45-07a7-41b6-9e00-28c489b71173-kube-api-access-ngkpj\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.251031 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.250977    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5v6r\" (UniqueName: \"kubernetes.io/projected/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-api-access-t5v6r\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.278523 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.278488    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"
Jul 24 03:21:56.438696 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.438654    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8"]
Jul 24 03:21:56.441818 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:21:56.441788    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2c90cb8_5136_428b_920b_0db0144e03ba.slice/crio-fbf93e466272efdf0c04c95fd9e48b4ad3bb43fe58e692b24dd07dede9f6bb62 WatchSource:0}: Error finding container fbf93e466272efdf0c04c95fd9e48b4ad3bb43fe58e692b24dd07dede9f6bb62: Status 404 returned error can't find the container with id fbf93e466272efdf0c04c95fd9e48b4ad3bb43fe58e692b24dd07dede9f6bb62
Jul 24 03:21:56.638068 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.638025    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8" event={"ID":"b2c90cb8-5136-428b-920b-0db0144e03ba","Type":"ContainerStarted","Data":"373db30d3492573898ef02e98223ddbe31f317bd07d3951f80c521d2780eaa25"}
Jul 24 03:21:56.638238 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.638088    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8" event={"ID":"b2c90cb8-5136-428b-920b-0db0144e03ba","Type":"ContainerStarted","Data":"c34eff846dcfa091e5c7b19b2d18220d47662cf7c683ba6a6aee96c67f8be369"}
Jul 24 03:21:56.638238 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.638109    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8" event={"ID":"b2c90cb8-5136-428b-920b-0db0144e03ba","Type":"ContainerStarted","Data":"fbf93e466272efdf0c04c95fd9e48b4ad3bb43fe58e692b24dd07dede9f6bb62"}
Jul 24 03:21:56.748006 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.747919    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.748180 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.748132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-tls\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.750820 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.750798    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/737a1d45-07a7-41b6-9e00-28c489b71173-node-exporter-tls\") pod \"node-exporter-2vsf6\" (UID: \"737a1d45-07a7-41b6-9e00-28c489b71173\") " pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.750980 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.750957    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/654d60b6-080a-4e5f-8a60-544e5986bf00-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-v9d9r\" (UID: \"654d60b6-080a-4e5f-8a60-544e5986bf00\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.901962 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.901924    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"
Jul 24 03:21:56.918458 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:56.918401    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2vsf6"
Jul 24 03:21:56.937801 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:21:56.937764    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod737a1d45_07a7_41b6_9e00_28c489b71173.slice/crio-d22a4a856d06e90604c9e3a776d46e56330b24e526e6ea7c16a6087d543ad56d WatchSource:0}: Error finding container d22a4a856d06e90604c9e3a776d46e56330b24e526e6ea7c16a6087d543ad56d: Status 404 returned error can't find the container with id d22a4a856d06e90604c9e3a776d46e56330b24e526e6ea7c16a6087d543ad56d
Jul 24 03:21:57.058804 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.057443    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 03:21:57.061704 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.061682    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.064487 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.064337    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 03:21:57.064749 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.064336    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-58l9w\""
Jul 24 03:21:57.065968 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.065146    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 03:21:57.065968 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.065395    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 03:21:57.065968 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.065613    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 03:21:57.065968 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.065826    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 03:21:57.067129 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.067107    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 03:21:57.069012 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.067364    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 03:21:57.069012 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.067415    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 03:21:57.069012 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.067457    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 03:21:57.075886 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.075785    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r"]
Jul 24 03:21:57.076215 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:21:57.076174    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod654d60b6_080a_4e5f_8a60_544e5986bf00.slice/crio-b9f7513f1db5540b1c4d7b1e1b8fb1ea7c9ed7a4dc56c0b9bfea5e69da9e220c WatchSource:0}: Error finding container b9f7513f1db5540b1c4d7b1e1b8fb1ea7c9ed7a4dc56c0b9bfea5e69da9e220c: Status 404 returned error can't find the container with id b9f7513f1db5540b1c4d7b1e1b8fb1ea7c9ed7a4dc56c0b9bfea5e69da9e220c
Jul 24 03:21:57.080292 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.080176    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 03:21:57.150784 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.150747    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.150963 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.150806    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.150963 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.150890    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.150963 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.150947    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a43140e9-18ba-44e5-ba7a-03979b7613cc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.150963 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.150976    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-web-config\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.150963 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.151007    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a43140e9-18ba-44e5-ba7a-03979b7613cc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.150963 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.151053    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-config-volume\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.151387 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.151107    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a43140e9-18ba-44e5-ba7a-03979b7613cc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.151387 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.151182    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a43140e9-18ba-44e5-ba7a-03979b7613cc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.151387 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.151250    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.151387 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.151276    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.151387 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.151300    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxpxq\" (UniqueName: \"kubernetes.io/projected/a43140e9-18ba-44e5-ba7a-03979b7613cc-kube-api-access-rxpxq\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.151387 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.151328    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a43140e9-18ba-44e5-ba7a-03979b7613cc-config-out\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.252377 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.252335    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a43140e9-18ba-44e5-ba7a-03979b7613cc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.252573 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.252418    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.252573 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.252469    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.252573 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.252492    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxpxq\" (UniqueName: \"kubernetes.io/projected/a43140e9-18ba-44e5-ba7a-03979b7613cc-kube-api-access-rxpxq\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.252573 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.252516    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a43140e9-18ba-44e5-ba7a-03979b7613cc-config-out\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.252573 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.252556    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.252834 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.252582    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.252834 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.252610    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.252834 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.252641    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a43140e9-18ba-44e5-ba7a-03979b7613cc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.252834 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.252659    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-web-config\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.252834 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.252681    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a43140e9-18ba-44e5-ba7a-03979b7613cc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.252834 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.252705    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-config-volume\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.252834 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.252752    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a43140e9-18ba-44e5-ba7a-03979b7613cc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.253854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.253166    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a43140e9-18ba-44e5-ba7a-03979b7613cc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.253854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.253619    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a43140e9-18ba-44e5-ba7a-03979b7613cc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.256477 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.256375    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a43140e9-18ba-44e5-ba7a-03979b7613cc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.256900 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.256855    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.257418 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.257379    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-web-config\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.258621 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.258538    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a43140e9-18ba-44e5-ba7a-03979b7613cc-config-out\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.259694 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.259647    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.259694 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.259685    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.260231 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.260182    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.260731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.260358    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-config-volume\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.260731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.260695    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a43140e9-18ba-44e5-ba7a-03979b7613cc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.261238 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.261212    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxpxq\" (UniqueName: \"kubernetes.io/projected/a43140e9-18ba-44e5-ba7a-03979b7613cc-kube-api-access-rxpxq\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.262715 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.262693    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.377538 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.377494    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:21:57.644626 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.644580    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r" event={"ID":"654d60b6-080a-4e5f-8a60-544e5986bf00","Type":"ContainerStarted","Data":"b9f7513f1db5540b1c4d7b1e1b8fb1ea7c9ed7a4dc56c0b9bfea5e69da9e220c"}
Jul 24 03:21:57.646031 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:21:57.645998    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2vsf6" event={"ID":"737a1d45-07a7-41b6-9e00-28c489b71173","Type":"ContainerStarted","Data":"d22a4a856d06e90604c9e3a776d46e56330b24e526e6ea7c16a6087d543ad56d"}
Jul 24 03:22:00.305414 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.304296    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"]
Jul 24 03:22:00.308114 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.308090    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.311529 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.311501    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 24 03:22:00.311761 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.311741    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2n6dk\""
Jul 24 03:22:00.312736 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.311942    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 24 03:22:00.312736 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.312155    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 24 03:22:00.312736 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.312337    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2hhr8bpmegh8e\""
Jul 24 03:22:00.312736 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.312578    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 24 03:22:00.317577 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.317549    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"]
Jul 24 03:22:00.385873 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.385837    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzjcw\" (UniqueName: \"kubernetes.io/projected/7fb24886-365d-4e28-b560-6fb915320fab-kube-api-access-fzjcw\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.386060 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.385899    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7fb24886-365d-4e28-b560-6fb915320fab-secret-metrics-server-tls\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.386060 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.385965    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7fb24886-365d-4e28-b560-6fb915320fab-client-ca-bundle\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.386060 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.386051    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7fb24886-365d-4e28-b560-6fb915320fab-audit-log\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.386210 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.386094    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7fb24886-365d-4e28-b560-6fb915320fab-metrics-server-audit-profiles\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.386210 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.386145    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fb24886-365d-4e28-b560-6fb915320fab-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.386210 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.386188    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7fb24886-365d-4e28-b560-6fb915320fab-secret-metrics-server-client-certs\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.487332 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.487287    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7fb24886-365d-4e28-b560-6fb915320fab-secret-metrics-server-tls\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.487539 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.487359    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7fb24886-365d-4e28-b560-6fb915320fab-client-ca-bundle\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.487539 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.487403    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7fb24886-365d-4e28-b560-6fb915320fab-audit-log\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.487539 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.487444    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7fb24886-365d-4e28-b560-6fb915320fab-metrics-server-audit-profiles\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.487539 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.487488    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fb24886-365d-4e28-b560-6fb915320fab-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.487539 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.487540    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7fb24886-365d-4e28-b560-6fb915320fab-secret-metrics-server-client-certs\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.487806 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.487580    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzjcw\" (UniqueName: \"kubernetes.io/projected/7fb24886-365d-4e28-b560-6fb915320fab-kube-api-access-fzjcw\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.488860 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.488527    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fb24886-365d-4e28-b560-6fb915320fab-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.488860 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.488724    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7fb24886-365d-4e28-b560-6fb915320fab-metrics-server-audit-profiles\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.488860 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.488817    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7fb24886-365d-4e28-b560-6fb915320fab-audit-log\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.490413 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.490385    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7fb24886-365d-4e28-b560-6fb915320fab-secret-metrics-server-tls\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.490695 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.490671    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7fb24886-365d-4e28-b560-6fb915320fab-client-ca-bundle\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.490877 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.490858    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7fb24886-365d-4e28-b560-6fb915320fab-secret-metrics-server-client-certs\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.495796 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.495774    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzjcw\" (UniqueName: \"kubernetes.io/projected/7fb24886-365d-4e28-b560-6fb915320fab-kube-api-access-fzjcw\") pod \"metrics-server-5b4fccfb64-hf8zj\" (UID: \"7fb24886-365d-4e28-b560-6fb915320fab\") " pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.622054 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.621956    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:00.756859 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.756830    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"]
Jul 24 03:22:00.760483 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.760458    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"
Jul 24 03:22:00.762549 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.762523    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 24 03:22:00.762680 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.762578    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xs8b6\""
Jul 24 03:22:00.770383 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.770055    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"]
Jul 24 03:22:00.892035 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.892001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/83b1ba72-f4aa-4d98-9382-65f711fbab75-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-cdtzk\" (UID: \"83b1ba72-f4aa-4d98-9382-65f711fbab75\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"
Jul 24 03:22:00.892186 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.892061    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44qwq\" (UniqueName: \"kubernetes.io/projected/83b1ba72-f4aa-4d98-9382-65f711fbab75-kube-api-access-44qwq\") pod \"monitoring-plugin-68b7fc9574-cdtzk\" (UID: \"83b1ba72-f4aa-4d98-9382-65f711fbab75\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"
Jul 24 03:22:00.992791 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.992753    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/83b1ba72-f4aa-4d98-9382-65f711fbab75-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-cdtzk\" (UID: \"83b1ba72-f4aa-4d98-9382-65f711fbab75\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"
Jul 24 03:22:00.992968 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:00.992819    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44qwq\" (UniqueName: \"kubernetes.io/projected/83b1ba72-f4aa-4d98-9382-65f711fbab75-kube-api-access-44qwq\") pod \"monitoring-plugin-68b7fc9574-cdtzk\" (UID: \"83b1ba72-f4aa-4d98-9382-65f711fbab75\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"
Jul 24 03:22:00.992968 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:22:00.992911    2574 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 24 03:22:00.993069 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:22:00.992975    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83b1ba72-f4aa-4d98-9382-65f711fbab75-monitoring-plugin-cert podName:83b1ba72-f4aa-4d98-9382-65f711fbab75 nodeName:}" failed. No retries permitted until 2026-07-24 03:22:01.492958015 +0000 UTC m=+276.456051285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/83b1ba72-f4aa-4d98-9382-65f711fbab75-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-cdtzk" (UID: "83b1ba72-f4aa-4d98-9382-65f711fbab75") : secret "monitoring-plugin-cert" not found
Jul 24 03:22:01.004314 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:01.004282    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44qwq\" (UniqueName: \"kubernetes.io/projected/83b1ba72-f4aa-4d98-9382-65f711fbab75-kube-api-access-44qwq\") pod \"monitoring-plugin-68b7fc9574-cdtzk\" (UID: \"83b1ba72-f4aa-4d98-9382-65f711fbab75\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"
Jul 24 03:22:01.498239 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:01.498204    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/83b1ba72-f4aa-4d98-9382-65f711fbab75-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-cdtzk\" (UID: \"83b1ba72-f4aa-4d98-9382-65f711fbab75\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"
Jul 24 03:22:01.501065 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:01.501037    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/83b1ba72-f4aa-4d98-9382-65f711fbab75-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-cdtzk\" (UID: \"83b1ba72-f4aa-4d98-9382-65f711fbab75\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"
Jul 24 03:22:01.674215 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:01.674183    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"
Jul 24 03:22:02.161206 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.161168    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 03:22:02.167832 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.167803    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.172382 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.172354    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 03:22:02.173032 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.172996    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 03:22:02.173146 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.173035    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 03:22:02.173146 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.173104    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 03:22:02.173146 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.173112    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 03:22:02.173146 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.173035    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 03:22:02.173293 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.173278    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 03:22:02.173397 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.173381    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 03:22:02.173756 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.173491    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k8vn5\""
Jul 24 03:22:02.174147 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.174128    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 03:22:02.174274 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.174252    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 03:22:02.174274 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.174262    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 03:22:02.174990 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.174272    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 03:22:02.174990 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.174213    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 03:22:02.181702 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.181678    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8ubppmkdidbep\""
Jul 24 03:22:02.186810 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.186578    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 03:22:02.194290 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.194262    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 03:22:02.307091 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307052    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307091 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307094    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307347 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307176    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-config\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307347 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307199    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307347 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307224    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1af0e672-5a9e-45f8-a3fd-44bc329168b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307347 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307241    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307347 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307317    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307637 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307356    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307637 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307383    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307637 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307408    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307637 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307451    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307637 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307475    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307637 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307493    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv6z5\" (UniqueName: \"kubernetes.io/projected/1af0e672-5a9e-45f8-a3fd-44bc329168b7-kube-api-access-nv6z5\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307637 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307526    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307637 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307555    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1af0e672-5a9e-45f8-a3fd-44bc329168b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307637 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307608    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.307637 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.308123 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307656    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.308123 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307681    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.308123 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.307703    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408499 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408458    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408691 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408507    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1af0e672-5a9e-45f8-a3fd-44bc329168b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408691 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408573    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408691 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408596    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408691 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408691 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408651    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408691 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408670    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408988 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408710    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408988 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408742    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408988 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408803    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-config\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408988 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408827    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408988 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408853    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1af0e672-5a9e-45f8-a3fd-44bc329168b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408988 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408877    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408988 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408925    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408988 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408950    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.408988 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.408977    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.409394 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.409000    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.409394 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.409026    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.409394 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.409057    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.409394 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.409084    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv6z5\" (UniqueName: \"kubernetes.io/projected/1af0e672-5a9e-45f8-a3fd-44bc329168b7-kube-api-access-nv6z5\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.410167 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.410073    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.410797 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.410717    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.412075 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.411340    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.412075 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.411870    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.413484 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.412678    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.413484 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.413097    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-config\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.413484 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.413301    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.413747 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.413508    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.415904 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.413809    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.416003 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.415984    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.416276 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.416251    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.416374 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.416307    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.416855 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.416813    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.419397 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.419352    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv6z5\" (UniqueName: \"kubernetes.io/projected/1af0e672-5a9e-45f8-a3fd-44bc329168b7-kube-api-access-nv6z5\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.421480 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.421360    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1af0e672-5a9e-45f8-a3fd-44bc329168b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.421617 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.421578    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.421770 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.421732    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.421878 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.421863    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1af0e672-5a9e-45f8-a3fd-44bc329168b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.422206 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.422182    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.422965 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.422942    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:02.488506 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:02.488464    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:06.320793 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.320649    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 03:22:06.326857 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:22:06.326796    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda43140e9_18ba_44e5_ba7a_03979b7613cc.slice/crio-53f481d0116ce2c23ec14ad294b3617be458e0c7ae851629c65dd05dc15eaa3e WatchSource:0}: Error finding container 53f481d0116ce2c23ec14ad294b3617be458e0c7ae851629c65dd05dc15eaa3e: Status 404 returned error can't find the container with id 53f481d0116ce2c23ec14ad294b3617be458e0c7ae851629c65dd05dc15eaa3e
Jul 24 03:22:06.544477 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.544210    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"]
Jul 24 03:22:06.548397 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.548047    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"]
Jul 24 03:22:06.549450 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:22:06.549368    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83b1ba72_f4aa_4d98_9382_65f711fbab75.slice/crio-e629fa823511b99252a7c56b9755d424fa83bf50518f0c82e3ada5a48518896e WatchSource:0}: Error finding container e629fa823511b99252a7c56b9755d424fa83bf50518f0c82e3ada5a48518896e: Status 404 returned error can't find the container with id e629fa823511b99252a7c56b9755d424fa83bf50518f0c82e3ada5a48518896e
Jul 24 03:22:06.551151 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:22:06.551113    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb24886_365d_4e28_b560_6fb915320fab.slice/crio-b7e616ad08f9c42e4131f12dad6d322aefcd9aee3026aad0ce82a7241d62dd5c WatchSource:0}: Error finding container b7e616ad08f9c42e4131f12dad6d322aefcd9aee3026aad0ce82a7241d62dd5c: Status 404 returned error can't find the container with id b7e616ad08f9c42e4131f12dad6d322aefcd9aee3026aad0ce82a7241d62dd5c
Jul 24 03:22:06.556387 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.556349    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 03:22:06.561465 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:22:06.561418    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1af0e672_5a9e_45f8_a3fd_44bc329168b7.slice/crio-54bec76a8e1a917b189ae44d8a928a07f3387104ad513d69f0c463bafcaa270c WatchSource:0}: Error finding container 54bec76a8e1a917b189ae44d8a928a07f3387104ad513d69f0c463bafcaa270c: Status 404 returned error can't find the container with id 54bec76a8e1a917b189ae44d8a928a07f3387104ad513d69f0c463bafcaa270c
Jul 24 03:22:06.690983 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.690936    2574 generic.go:356] "Generic (PLEG): container finished" podID="737a1d45-07a7-41b6-9e00-28c489b71173" containerID="73ef45a52c46c95a94e0fb6a6f3747cc850805ff91d36b87abf8df6f059da5ea" exitCode=0
Jul 24 03:22:06.691170 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.691025    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2vsf6" event={"ID":"737a1d45-07a7-41b6-9e00-28c489b71173","Type":"ContainerDied","Data":"73ef45a52c46c95a94e0fb6a6f3747cc850805ff91d36b87abf8df6f059da5ea"}
Jul 24 03:22:06.692466 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.692412    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerStarted","Data":"54bec76a8e1a917b189ae44d8a928a07f3387104ad513d69f0c463bafcaa270c"}
Jul 24 03:22:06.693990 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.693961    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj" event={"ID":"7fb24886-365d-4e28-b560-6fb915320fab","Type":"ContainerStarted","Data":"b7e616ad08f9c42e4131f12dad6d322aefcd9aee3026aad0ce82a7241d62dd5c"}
Jul 24 03:22:06.697124 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.697092    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8" event={"ID":"b2c90cb8-5136-428b-920b-0db0144e03ba","Type":"ContainerStarted","Data":"cefe387bc07e8556b7837c474878be19caa3fb8bc7d78fd1d925a9c60858f436"}
Jul 24 03:22:06.705809 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.705771    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r" event={"ID":"654d60b6-080a-4e5f-8a60-544e5986bf00","Type":"ContainerStarted","Data":"007ebf226da2c56ecaf2191f21b35bca264d2e0c346986094a59a24b78160696"}
Jul 24 03:22:06.705973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.705832    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r" event={"ID":"654d60b6-080a-4e5f-8a60-544e5986bf00","Type":"ContainerStarted","Data":"b640beb202b7ff332aa1dce0b65c8bc82c6192769281b50ed3d9b4bb0cf5649e"}
Jul 24 03:22:06.705973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.705854    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r" event={"ID":"654d60b6-080a-4e5f-8a60-544e5986bf00","Type":"ContainerStarted","Data":"e7b76d4dae3d02a9ee87278cff0c5fb97fd5271ffb501e5c838d366c3f5df88c"}
Jul 24 03:22:06.715180 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.714885    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk" event={"ID":"83b1ba72-f4aa-4d98-9382-65f711fbab75","Type":"ContainerStarted","Data":"e629fa823511b99252a7c56b9755d424fa83bf50518f0c82e3ada5a48518896e"}
Jul 24 03:22:06.716782 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.716736    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-48xzj" event={"ID":"985df3af-4c44-462b-90e2-46bbe9a6b181","Type":"ContainerStarted","Data":"6654c1aad792a24f8bf39eeaf4b46cf3b51baeb06eedcc976dbd6a81e3246370"}
Jul 24 03:22:06.716982 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.716877    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-48xzj"
Jul 24 03:22:06.719595 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.719572    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerStarted","Data":"53f481d0116ce2c23ec14ad294b3617be458e0c7ae851629c65dd05dc15eaa3e"}
Jul 24 03:22:06.725731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.725703    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-48xzj"
Jul 24 03:22:06.730249 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.730201    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-v9d9r" podStartSLOduration=2.647019781 podStartE2EDuration="11.730181766s" totalImagesPullingTime="9.083161985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:06.728068802 +0000 UTC m=+281.691162091" watchObservedRunningTime="2026-07-24 03:22:06.730181766 +0000 UTC m=+281.693275051"
Jul 24 03:22:06.746141 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.746079    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-q7nq8" podStartSLOduration=2.205356766 podStartE2EDuration="11.746054871s" totalImagesPullingTime="9.540698105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:06.745015125 +0000 UTC m=+281.708108446" watchObservedRunningTime="2026-07-24 03:22:06.746054871 +0000 UTC m=+281.709148162"
Jul 24 03:22:06.762698 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:06.762632    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-48xzj" podStartSLOduration=3.35002672 podStartE2EDuration="21.762605912s" totalImagesPullingTime="18.412579192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:06.761483823 +0000 UTC m=+281.724577110" watchObservedRunningTime="2026-07-24 03:22:06.762605912 +0000 UTC m=+281.725699200"
Jul 24 03:22:07.276338 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:22:07.276291    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tb9dz" podUID="3d10c276-6ba4-44b2-9895-1bb803660d4e"
Jul 24 03:22:07.736017 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:07.735977    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:22:07.740573 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:07.739221    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2vsf6" event={"ID":"737a1d45-07a7-41b6-9e00-28c489b71173","Type":"ContainerStarted","Data":"99a691885a88cd49e493bf8339f97a3c078606207f15a7abe29a8767b1f0b8a1"}
Jul 24 03:22:07.740573 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:07.739326    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2vsf6" event={"ID":"737a1d45-07a7-41b6-9e00-28c489b71173","Type":"ContainerStarted","Data":"da6edb20ae9e4fd333c8bf6093d735e9763e2fdbc7812d9dc26c32cf3b02996b"}
Jul 24 03:22:07.766112 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:07.764547    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2vsf6" podStartSLOduration=3.481557762 podStartE2EDuration="12.764524245s" totalImagesPullingTime="9.165190717s" totalInitContainerRuntime="117.775766ms" isStatefulPod=true podCreationTimestamp="2026-07-24 03:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:07.763362375 +0000 UTC m=+282.726455665" watchObservedRunningTime="2026-07-24 03:22:07.764524245 +0000 UTC m=+282.727617534"
Jul 24 03:22:10.497725 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.497687    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:22:10.502836 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.501775    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d10c276-6ba4-44b2-9895-1bb803660d4e-metrics-tls\") pod \"dns-default-tb9dz\" (UID: \"3d10c276-6ba4-44b2-9895-1bb803660d4e\") " pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:22:10.740115 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.740032    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fzmf8\""
Jul 24 03:22:10.748224 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.747996    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:22:10.754840 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.754404    2574 generic.go:356] "Generic (PLEG): container finished" podID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerID="ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2" exitCode=0
Jul 24 03:22:10.754840 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.754520    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerDied","Data":"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"}
Jul 24 03:22:10.757045 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.757010    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj" event={"ID":"7fb24886-365d-4e28-b560-6fb915320fab","Type":"ContainerStarted","Data":"66b72f7bc82b22e480ce8ea6424d4b34d9d24f57ba7fa46f81df4c6878209230"}
Jul 24 03:22:10.759716 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.759684    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk" event={"ID":"83b1ba72-f4aa-4d98-9382-65f711fbab75","Type":"ContainerStarted","Data":"cf90d2ab77cb40facbb54f2b1a87f7ef811d147de4b04486372232deee0a9b71"}
Jul 24 03:22:10.760034 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.760014    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"
Jul 24 03:22:10.761705 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.761677    2574 generic.go:356] "Generic (PLEG): container finished" podID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerID="a4e5361c26385a0c7615c7df337ebaa69e3c81c4732e2107b8af0b9b9a32ae83" exitCode=0
Jul 24 03:22:10.761825 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.761767    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerDied","Data":"a4e5361c26385a0c7615c7df337ebaa69e3c81c4732e2107b8af0b9b9a32ae83"}
Jul 24 03:22:10.765597 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.765548    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk"
Jul 24 03:22:10.813264 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.813204    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj" podStartSLOduration=7.120357202 podStartE2EDuration="10.8131833s" totalImagesPullingTime="3.692826098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:22:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:10.811853771 +0000 UTC m=+285.774947072" watchObservedRunningTime="2026-07-24 03:22:10.8131833 +0000 UTC m=+285.776276659"
Jul 24 03:22:10.896877 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.896817    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-cdtzk" podStartSLOduration=7.20650469 podStartE2EDuration="10.896793928s" totalImagesPullingTime="3.690289238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:22:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:10.895897057 +0000 UTC m=+285.858990347" watchObservedRunningTime="2026-07-24 03:22:10.896793928 +0000 UTC m=+285.859887218"
Jul 24 03:22:10.999961 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:10.999855    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tb9dz"]
Jul 24 03:22:11.005193 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:22:11.005165    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d10c276_6ba4_44b2_9895_1bb803660d4e.slice/crio-c6dc46978e066cb4c71d77494d9b278190c759b5477b5232963b0a656c3b96dc WatchSource:0}: Error finding container c6dc46978e066cb4c71d77494d9b278190c759b5477b5232963b0a656c3b96dc: Status 404 returned error can't find the container with id c6dc46978e066cb4c71d77494d9b278190c759b5477b5232963b0a656c3b96dc
Jul 24 03:22:11.779882 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:11.779836    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tb9dz" event={"ID":"3d10c276-6ba4-44b2-9895-1bb803660d4e","Type":"ContainerStarted","Data":"c6dc46978e066cb4c71d77494d9b278190c759b5477b5232963b0a656c3b96dc"}
Jul 24 03:22:16.810962 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:16.810921    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tb9dz" event={"ID":"3d10c276-6ba4-44b2-9895-1bb803660d4e","Type":"ContainerStarted","Data":"bfe67e50586e362bda15f8c104899910549a02548224cf20472ee745cfade1be"}
Jul 24 03:22:16.811519 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:16.810994    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tb9dz" event={"ID":"3d10c276-6ba4-44b2-9895-1bb803660d4e","Type":"ContainerStarted","Data":"bfb7e0c500c2310062330f4af72b9a39b273efbbaab13d582a4af38cef02a7c4"}
Jul 24 03:22:16.814258 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:16.814227    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerStarted","Data":"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"}
Jul 24 03:22:16.814419 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:16.814306    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerStarted","Data":"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"}
Jul 24 03:22:16.818809 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:16.818780    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerStarted","Data":"4595ecd4f1994364a08ffde45313376b56685e8ab6e0710bf33d657f7933df60"}
Jul 24 03:22:16.818931 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:16.818846    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerStarted","Data":"720c60fd860faf0569aef95ad8d7b019b9a038759b3a08a684325df01cf31788"}
Jul 24 03:22:16.818931 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:16.818891    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerStarted","Data":"469fd0789e5fd5f28a2e0b9d82a732f18169a751892073a49d408895a86d5f85"}
Jul 24 03:22:16.819002 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:16.818931    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerStarted","Data":"3e12a36671aa9c60f8c4fc3e8863b5fc4f37f6dda5c13ab8206b564b5e5911e9"}
Jul 24 03:22:16.819002 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:16.818970    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerStarted","Data":"8c2abfa484b549b40291d2ec406b69fcc4e528c2612e38a399bbf953486e50a8"}
Jul 24 03:22:16.831539 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:16.831451    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tb9dz" podStartSLOduration=252.020010089 podStartE2EDuration="4m16.831412199s" totalImagesPullingTime="4.81140211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:16.829372232 +0000 UTC m=+291.792465532" watchObservedRunningTime="2026-07-24 03:22:16.831412199 +0000 UTC m=+291.794505487"
Jul 24 03:22:17.824020 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:17.823989    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:22:18.831171 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:18.831138    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerStarted","Data":"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"}
Jul 24 03:22:18.831585 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:18.831261    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerStarted","Data":"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"}
Jul 24 03:22:18.831585 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:18.831294    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerStarted","Data":"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"}
Jul 24 03:22:18.834440 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:18.834390    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerStarted","Data":"bd5869a4221263781603b440036e46b8ad1c80bfe1bf21771ba2a8a85164c65b"}
Jul 24 03:22:18.861862 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:18.861801    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=11.19275614 podStartE2EDuration="21.861779113s" totalImagesPullingTime="10.648607837s" totalInitContainerRuntime="20.415136ms" isStatefulPod=false podCreationTimestamp="2026-07-24 03:21:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:18.858337513 +0000 UTC m=+293.821430799" watchObservedRunningTime="2026-07-24 03:22:18.861779113 +0000 UTC m=+293.824872402"
Jul 24 03:22:19.842579 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:19.842540    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerStarted","Data":"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"}
Jul 24 03:22:19.876658 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:19.876552    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.765493409 podStartE2EDuration="17.876528758s" totalImagesPullingTime="11.064511195s" totalInitContainerRuntime="46.524154ms" isStatefulPod=false podCreationTimestamp="2026-07-24 03:22:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:22:19.875130073 +0000 UTC m=+294.838223371" watchObservedRunningTime="2026-07-24 03:22:19.876528758 +0000 UTC m=+294.839622046"
Jul 24 03:22:20.622718 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:20.622677    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:20.622895 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:20.622742    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:22.489585 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:22.489546    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:22:25.423952 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:25.423921    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:22:25.425466 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:25.425441    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:22:25.435833 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:25.435815    2574 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 03:22:27.837480 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:27.837367    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tb9dz"
Jul 24 03:22:27.880498 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:27.880461    2574 generic.go:356] "Generic (PLEG): container finished" podID="bc7cb98c-bd37-44bf-80f0-1cd615e8a36f" containerID="d85cfcda00386fb90c4670651236c953d7615aae1f82b83fc7a05f016089547a" exitCode=0
Jul 24 03:22:27.880659 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:27.880537    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp" event={"ID":"bc7cb98c-bd37-44bf-80f0-1cd615e8a36f","Type":"ContainerDied","Data":"d85cfcda00386fb90c4670651236c953d7615aae1f82b83fc7a05f016089547a"}
Jul 24 03:22:27.880880 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:27.880864    2574 scope.go:117] "RemoveContainer" containerID="d85cfcda00386fb90c4670651236c953d7615aae1f82b83fc7a05f016089547a"
Jul 24 03:22:28.886330 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:28.886284    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp" event={"ID":"bc7cb98c-bd37-44bf-80f0-1cd615e8a36f","Type":"ContainerStarted","Data":"8832dacbe44cb28cfc1f104d437ab5101bd6969a85d02b44b6233fcd99939922"}
Jul 24 03:22:33.974376 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:33.974348    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tb9dz_3d10c276-6ba4-44b2-9895-1bb803660d4e/dns/0.log"
Jul 24 03:22:33.979761 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:33.979722    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tb9dz_3d10c276-6ba4-44b2-9895-1bb803660d4e/kube-rbac-proxy/0.log"
Jul 24 03:22:34.253258 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:34.253183    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s7jjg_559deb09-e606-40bb-ba17-b957dca7b41a/dns-node-resolver/0.log"
Jul 24 03:22:40.628667 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:40.628634    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:22:40.632694 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:22:40.632668    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5b4fccfb64-hf8zj"
Jul 24 03:23:02.488710 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:02.488649    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:02.505572 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:02.505547    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:03.055141 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:03.055112    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:16.197731 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:16.197653    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 03:23:16.198210 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:16.197978    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="alertmanager" containerID="cri-o://8c2abfa484b549b40291d2ec406b69fcc4e528c2612e38a399bbf953486e50a8" gracePeriod=120
Jul 24 03:23:16.198210 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:16.198034    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="kube-rbac-proxy-metric" containerID="cri-o://4595ecd4f1994364a08ffde45313376b56685e8ab6e0710bf33d657f7933df60" gracePeriod=120
Jul 24 03:23:16.198210 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:16.198060    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="prom-label-proxy" containerID="cri-o://bd5869a4221263781603b440036e46b8ad1c80bfe1bf21771ba2a8a85164c65b" gracePeriod=120
Jul 24 03:23:16.198210 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:16.198084    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="kube-rbac-proxy" containerID="cri-o://720c60fd860faf0569aef95ad8d7b019b9a038759b3a08a684325df01cf31788" gracePeriod=120
Jul 24 03:23:16.198210 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:16.198093    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="kube-rbac-proxy-web" containerID="cri-o://469fd0789e5fd5f28a2e0b9d82a732f18169a751892073a49d408895a86d5f85" gracePeriod=120
Jul 24 03:23:16.198210 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:16.198120    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="config-reloader" containerID="cri-o://3e12a36671aa9c60f8c4fc3e8863b5fc4f37f6dda5c13ab8206b564b5e5911e9" gracePeriod=120
Jul 24 03:23:17.107924 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.107889    2574 generic.go:356] "Generic (PLEG): container finished" podID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerID="bd5869a4221263781603b440036e46b8ad1c80bfe1bf21771ba2a8a85164c65b" exitCode=0
Jul 24 03:23:17.107924 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.107919    2574 generic.go:356] "Generic (PLEG): container finished" podID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerID="4595ecd4f1994364a08ffde45313376b56685e8ab6e0710bf33d657f7933df60" exitCode=0
Jul 24 03:23:17.107924 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.107929    2574 generic.go:356] "Generic (PLEG): container finished" podID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerID="720c60fd860faf0569aef95ad8d7b019b9a038759b3a08a684325df01cf31788" exitCode=0
Jul 24 03:23:17.107924 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.107941    2574 generic.go:356] "Generic (PLEG): container finished" podID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerID="3e12a36671aa9c60f8c4fc3e8863b5fc4f37f6dda5c13ab8206b564b5e5911e9" exitCode=0
Jul 24 03:23:17.108253 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.107946    2574 generic.go:356] "Generic (PLEG): container finished" podID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerID="8c2abfa484b549b40291d2ec406b69fcc4e528c2612e38a399bbf953486e50a8" exitCode=0
Jul 24 03:23:17.108253 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.107951    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerDied","Data":"bd5869a4221263781603b440036e46b8ad1c80bfe1bf21771ba2a8a85164c65b"}
Jul 24 03:23:17.108253 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.108039    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerDied","Data":"4595ecd4f1994364a08ffde45313376b56685e8ab6e0710bf33d657f7933df60"}
Jul 24 03:23:17.108253 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.108068    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerDied","Data":"720c60fd860faf0569aef95ad8d7b019b9a038759b3a08a684325df01cf31788"}
Jul 24 03:23:17.108253 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.108097    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerDied","Data":"3e12a36671aa9c60f8c4fc3e8863b5fc4f37f6dda5c13ab8206b564b5e5911e9"}
Jul 24 03:23:17.108253 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.108120    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerDied","Data":"8c2abfa484b549b40291d2ec406b69fcc4e528c2612e38a399bbf953486e50a8"}
Jul 24 03:23:17.454836 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.454810    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:17.535784 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.535749    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-web-config\") pod \"a43140e9-18ba-44e5-ba7a-03979b7613cc\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") "
Jul 24 03:23:17.535784 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.535786    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a43140e9-18ba-44e5-ba7a-03979b7613cc-tls-assets\") pod \"a43140e9-18ba-44e5-ba7a-03979b7613cc\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") "
Jul 24 03:23:17.536033 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.535814    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a43140e9-18ba-44e5-ba7a-03979b7613cc-metrics-client-ca\") pod \"a43140e9-18ba-44e5-ba7a-03979b7613cc\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") "
Jul 24 03:23:17.536033 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.535839    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a43140e9-18ba-44e5-ba7a-03979b7613cc-config-out\") pod \"a43140e9-18ba-44e5-ba7a-03979b7613cc\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") "
Jul 24 03:23:17.536033 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.535857    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-cluster-tls-config\") pod \"a43140e9-18ba-44e5-ba7a-03979b7613cc\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") "
Jul 24 03:23:17.536033 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.535895    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-main-tls\") pod \"a43140e9-18ba-44e5-ba7a-03979b7613cc\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") "
Jul 24 03:23:17.536033 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.535910    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxpxq\" (UniqueName: \"kubernetes.io/projected/a43140e9-18ba-44e5-ba7a-03979b7613cc-kube-api-access-rxpxq\") pod \"a43140e9-18ba-44e5-ba7a-03979b7613cc\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") "
Jul 24 03:23:17.536033 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.535937    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-config-volume\") pod \"a43140e9-18ba-44e5-ba7a-03979b7613cc\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") "
Jul 24 03:23:17.536033 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.535960    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy\") pod \"a43140e9-18ba-44e5-ba7a-03979b7613cc\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") "
Jul 24 03:23:17.536033 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.536030    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a43140e9-18ba-44e5-ba7a-03979b7613cc-alertmanager-main-db\") pod \"a43140e9-18ba-44e5-ba7a-03979b7613cc\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") "
Jul 24 03:23:17.536621 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.536065    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a43140e9-18ba-44e5-ba7a-03979b7613cc\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") "
Jul 24 03:23:17.536621 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.536104    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"a43140e9-18ba-44e5-ba7a-03979b7613cc\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") "
Jul 24 03:23:17.536621 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.536140    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a43140e9-18ba-44e5-ba7a-03979b7613cc-alertmanager-trusted-ca-bundle\") pod \"a43140e9-18ba-44e5-ba7a-03979b7613cc\" (UID: \"a43140e9-18ba-44e5-ba7a-03979b7613cc\") "
Jul 24 03:23:17.536621 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.536213    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a43140e9-18ba-44e5-ba7a-03979b7613cc-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a43140e9-18ba-44e5-ba7a-03979b7613cc" (UID: "a43140e9-18ba-44e5-ba7a-03979b7613cc"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:23:17.537217 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.536733    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a43140e9-18ba-44e5-ba7a-03979b7613cc-metrics-client-ca\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:17.537217 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.536822    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a43140e9-18ba-44e5-ba7a-03979b7613cc-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a43140e9-18ba-44e5-ba7a-03979b7613cc" (UID: "a43140e9-18ba-44e5-ba7a-03979b7613cc"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:23:17.537217 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.537042    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a43140e9-18ba-44e5-ba7a-03979b7613cc-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a43140e9-18ba-44e5-ba7a-03979b7613cc" (UID: "a43140e9-18ba-44e5-ba7a-03979b7613cc"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:23:17.539735 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.539686    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a43140e9-18ba-44e5-ba7a-03979b7613cc" (UID: "a43140e9-18ba-44e5-ba7a-03979b7613cc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:17.539735 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.539706    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a43140e9-18ba-44e5-ba7a-03979b7613cc-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a43140e9-18ba-44e5-ba7a-03979b7613cc" (UID: "a43140e9-18ba-44e5-ba7a-03979b7613cc"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:23:17.539932 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.539818    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-config-volume" (OuterVolumeSpecName: "config-volume") pod "a43140e9-18ba-44e5-ba7a-03979b7613cc" (UID: "a43140e9-18ba-44e5-ba7a-03979b7613cc"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:17.539932 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.539908    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a43140e9-18ba-44e5-ba7a-03979b7613cc-config-out" (OuterVolumeSpecName: "config-out") pod "a43140e9-18ba-44e5-ba7a-03979b7613cc" (UID: "a43140e9-18ba-44e5-ba7a-03979b7613cc"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:23:17.540011 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.539989    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a43140e9-18ba-44e5-ba7a-03979b7613cc" (UID: "a43140e9-18ba-44e5-ba7a-03979b7613cc"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:17.540507 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.540470    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a43140e9-18ba-44e5-ba7a-03979b7613cc" (UID: "a43140e9-18ba-44e5-ba7a-03979b7613cc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:17.541039 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.541015    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a43140e9-18ba-44e5-ba7a-03979b7613cc-kube-api-access-rxpxq" (OuterVolumeSpecName: "kube-api-access-rxpxq") pod "a43140e9-18ba-44e5-ba7a-03979b7613cc" (UID: "a43140e9-18ba-44e5-ba7a-03979b7613cc"). InnerVolumeSpecName "kube-api-access-rxpxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:23:17.541252 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.541234    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a43140e9-18ba-44e5-ba7a-03979b7613cc" (UID: "a43140e9-18ba-44e5-ba7a-03979b7613cc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:17.544115 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.544094    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a43140e9-18ba-44e5-ba7a-03979b7613cc" (UID: "a43140e9-18ba-44e5-ba7a-03979b7613cc"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:17.552058 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.552029    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-web-config" (OuterVolumeSpecName: "web-config") pod "a43140e9-18ba-44e5-ba7a-03979b7613cc" (UID: "a43140e9-18ba-44e5-ba7a-03979b7613cc"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:17.637870 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.637840    2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a43140e9-18ba-44e5-ba7a-03979b7613cc-tls-assets\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:17.637870 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.637871    2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a43140e9-18ba-44e5-ba7a-03979b7613cc-config-out\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:17.638052 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.637880    2574 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-cluster-tls-config\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:17.638052 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.637890    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-main-tls\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:17.638052 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.637898    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxpxq\" (UniqueName: \"kubernetes.io/projected/a43140e9-18ba-44e5-ba7a-03979b7613cc-kube-api-access-rxpxq\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:17.638052 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.637906    2574 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:17.638052 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.637916    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:17.638052 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.637925    2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a43140e9-18ba-44e5-ba7a-03979b7613cc-alertmanager-main-db\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:17.638052 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.637934    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:17.638052 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.637946    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:17.638052 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.637954    2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a43140e9-18ba-44e5-ba7a-03979b7613cc-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:17.638052 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:17.637985    2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a43140e9-18ba-44e5-ba7a-03979b7613cc-web-config\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:18.115628 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.115537    2574 generic.go:356] "Generic (PLEG): container finished" podID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerID="469fd0789e5fd5f28a2e0b9d82a732f18169a751892073a49d408895a86d5f85" exitCode=0
Jul 24 03:23:18.115628 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.115615    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 03:23:18.115628 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.115617    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerDied","Data":"469fd0789e5fd5f28a2e0b9d82a732f18169a751892073a49d408895a86d5f85"}
Jul 24 03:23:18.115908 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.115706    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a43140e9-18ba-44e5-ba7a-03979b7613cc","Type":"ContainerDied","Data":"53f481d0116ce2c23ec14ad294b3617be458e0c7ae851629c65dd05dc15eaa3e"}
Jul 24 03:23:18.115908 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.115755    2574 scope.go:117] "RemoveContainer" containerID="bd5869a4221263781603b440036e46b8ad1c80bfe1bf21771ba2a8a85164c65b"
Jul 24 03:23:18.125243 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.125087    2574 scope.go:117] "RemoveContainer" containerID="4595ecd4f1994364a08ffde45313376b56685e8ab6e0710bf33d657f7933df60"
Jul 24 03:23:18.134498 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.134471    2574 scope.go:117] "RemoveContainer" containerID="720c60fd860faf0569aef95ad8d7b019b9a038759b3a08a684325df01cf31788"
Jul 24 03:23:18.140494 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.140464    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 03:23:18.144602 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.144337    2574 scope.go:117] "RemoveContainer" containerID="469fd0789e5fd5f28a2e0b9d82a732f18169a751892073a49d408895a86d5f85"
Jul 24 03:23:18.146452 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.146411    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 03:23:18.152780 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.152758    2574 scope.go:117] "RemoveContainer" containerID="3e12a36671aa9c60f8c4fc3e8863b5fc4f37f6dda5c13ab8206b564b5e5911e9"
Jul 24 03:23:18.160837 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.160816    2574 scope.go:117] "RemoveContainer" containerID="8c2abfa484b549b40291d2ec406b69fcc4e528c2612e38a399bbf953486e50a8"
Jul 24 03:23:18.169370 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.169347    2574 scope.go:117] "RemoveContainer" containerID="a4e5361c26385a0c7615c7df337ebaa69e3c81c4732e2107b8af0b9b9a32ae83"
Jul 24 03:23:18.177897 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.177868    2574 scope.go:117] "RemoveContainer" containerID="bd5869a4221263781603b440036e46b8ad1c80bfe1bf21771ba2a8a85164c65b"
Jul 24 03:23:18.178267 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:18.178240    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd5869a4221263781603b440036e46b8ad1c80bfe1bf21771ba2a8a85164c65b\": container with ID starting with bd5869a4221263781603b440036e46b8ad1c80bfe1bf21771ba2a8a85164c65b not found: ID does not exist" containerID="bd5869a4221263781603b440036e46b8ad1c80bfe1bf21771ba2a8a85164c65b"
Jul 24 03:23:18.178404 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.178279    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd5869a4221263781603b440036e46b8ad1c80bfe1bf21771ba2a8a85164c65b"} err="failed to get container status \"bd5869a4221263781603b440036e46b8ad1c80bfe1bf21771ba2a8a85164c65b\": rpc error: code = NotFound desc = could not find container \"bd5869a4221263781603b440036e46b8ad1c80bfe1bf21771ba2a8a85164c65b\": container with ID starting with bd5869a4221263781603b440036e46b8ad1c80bfe1bf21771ba2a8a85164c65b not found: ID does not exist"
Jul 24 03:23:18.178404 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.178324    2574 scope.go:117] "RemoveContainer" containerID="4595ecd4f1994364a08ffde45313376b56685e8ab6e0710bf33d657f7933df60"
Jul 24 03:23:18.178661 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:18.178640    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4595ecd4f1994364a08ffde45313376b56685e8ab6e0710bf33d657f7933df60\": container with ID starting with 4595ecd4f1994364a08ffde45313376b56685e8ab6e0710bf33d657f7933df60 not found: ID does not exist" containerID="4595ecd4f1994364a08ffde45313376b56685e8ab6e0710bf33d657f7933df60"
Jul 24 03:23:18.178766 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.178665    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4595ecd4f1994364a08ffde45313376b56685e8ab6e0710bf33d657f7933df60"} err="failed to get container status \"4595ecd4f1994364a08ffde45313376b56685e8ab6e0710bf33d657f7933df60\": rpc error: code = NotFound desc = could not find container \"4595ecd4f1994364a08ffde45313376b56685e8ab6e0710bf33d657f7933df60\": container with ID starting with 4595ecd4f1994364a08ffde45313376b56685e8ab6e0710bf33d657f7933df60 not found: ID does not exist"
Jul 24 03:23:18.178766 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.178681    2574 scope.go:117] "RemoveContainer" containerID="720c60fd860faf0569aef95ad8d7b019b9a038759b3a08a684325df01cf31788"
Jul 24 03:23:18.179203 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:18.178981    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"720c60fd860faf0569aef95ad8d7b019b9a038759b3a08a684325df01cf31788\": container with ID starting with 720c60fd860faf0569aef95ad8d7b019b9a038759b3a08a684325df01cf31788 not found: ID does not exist" containerID="720c60fd860faf0569aef95ad8d7b019b9a038759b3a08a684325df01cf31788"
Jul 24 03:23:18.179203 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.179013    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"720c60fd860faf0569aef95ad8d7b019b9a038759b3a08a684325df01cf31788"} err="failed to get container status \"720c60fd860faf0569aef95ad8d7b019b9a038759b3a08a684325df01cf31788\": rpc error: code = NotFound desc = could not find container \"720c60fd860faf0569aef95ad8d7b019b9a038759b3a08a684325df01cf31788\": container with ID starting with 720c60fd860faf0569aef95ad8d7b019b9a038759b3a08a684325df01cf31788 not found: ID does not exist"
Jul 24 03:23:18.179203 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.179033    2574 scope.go:117] "RemoveContainer" containerID="469fd0789e5fd5f28a2e0b9d82a732f18169a751892073a49d408895a86d5f85"
Jul 24 03:23:18.179906 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:18.179318    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"469fd0789e5fd5f28a2e0b9d82a732f18169a751892073a49d408895a86d5f85\": container with ID starting with 469fd0789e5fd5f28a2e0b9d82a732f18169a751892073a49d408895a86d5f85 not found: ID does not exist" containerID="469fd0789e5fd5f28a2e0b9d82a732f18169a751892073a49d408895a86d5f85"
Jul 24 03:23:18.179906 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.179347    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"469fd0789e5fd5f28a2e0b9d82a732f18169a751892073a49d408895a86d5f85"} err="failed to get container status \"469fd0789e5fd5f28a2e0b9d82a732f18169a751892073a49d408895a86d5f85\": rpc error: code = NotFound desc = could not find container \"469fd0789e5fd5f28a2e0b9d82a732f18169a751892073a49d408895a86d5f85\": container with ID starting with 469fd0789e5fd5f28a2e0b9d82a732f18169a751892073a49d408895a86d5f85 not found: ID does not exist"
Jul 24 03:23:18.179906 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.179367    2574 scope.go:117] "RemoveContainer" containerID="3e12a36671aa9c60f8c4fc3e8863b5fc4f37f6dda5c13ab8206b564b5e5911e9"
Jul 24 03:23:18.179906 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:18.179664    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e12a36671aa9c60f8c4fc3e8863b5fc4f37f6dda5c13ab8206b564b5e5911e9\": container with ID starting with 3e12a36671aa9c60f8c4fc3e8863b5fc4f37f6dda5c13ab8206b564b5e5911e9 not found: ID does not exist" containerID="3e12a36671aa9c60f8c4fc3e8863b5fc4f37f6dda5c13ab8206b564b5e5911e9"
Jul 24 03:23:18.179906 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.179688    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e12a36671aa9c60f8c4fc3e8863b5fc4f37f6dda5c13ab8206b564b5e5911e9"} err="failed to get container status \"3e12a36671aa9c60f8c4fc3e8863b5fc4f37f6dda5c13ab8206b564b5e5911e9\": rpc error: code = NotFound desc = could not find container \"3e12a36671aa9c60f8c4fc3e8863b5fc4f37f6dda5c13ab8206b564b5e5911e9\": container with ID starting with 3e12a36671aa9c60f8c4fc3e8863b5fc4f37f6dda5c13ab8206b564b5e5911e9 not found: ID does not exist"
Jul 24 03:23:18.179906 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.179705    2574 scope.go:117] "RemoveContainer" containerID="8c2abfa484b549b40291d2ec406b69fcc4e528c2612e38a399bbf953486e50a8"
Jul 24 03:23:18.180896 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:18.180875    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c2abfa484b549b40291d2ec406b69fcc4e528c2612e38a399bbf953486e50a8\": container with ID starting with 8c2abfa484b549b40291d2ec406b69fcc4e528c2612e38a399bbf953486e50a8 not found: ID does not exist" containerID="8c2abfa484b549b40291d2ec406b69fcc4e528c2612e38a399bbf953486e50a8"
Jul 24 03:23:18.181003 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.180903    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c2abfa484b549b40291d2ec406b69fcc4e528c2612e38a399bbf953486e50a8"} err="failed to get container status \"8c2abfa484b549b40291d2ec406b69fcc4e528c2612e38a399bbf953486e50a8\": rpc error: code = NotFound desc = could not find container \"8c2abfa484b549b40291d2ec406b69fcc4e528c2612e38a399bbf953486e50a8\": container with ID starting with 8c2abfa484b549b40291d2ec406b69fcc4e528c2612e38a399bbf953486e50a8 not found: ID does not exist"
Jul 24 03:23:18.181003 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.180921    2574 scope.go:117] "RemoveContainer" containerID="a4e5361c26385a0c7615c7df337ebaa69e3c81c4732e2107b8af0b9b9a32ae83"
Jul 24 03:23:18.181246 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:18.181217    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4e5361c26385a0c7615c7df337ebaa69e3c81c4732e2107b8af0b9b9a32ae83\": container with ID starting with a4e5361c26385a0c7615c7df337ebaa69e3c81c4732e2107b8af0b9b9a32ae83 not found: ID does not exist" containerID="a4e5361c26385a0c7615c7df337ebaa69e3c81c4732e2107b8af0b9b9a32ae83"
Jul 24 03:23:18.181311 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:18.181242    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4e5361c26385a0c7615c7df337ebaa69e3c81c4732e2107b8af0b9b9a32ae83"} err="failed to get container status \"a4e5361c26385a0c7615c7df337ebaa69e3c81c4732e2107b8af0b9b9a32ae83\": rpc error: code = NotFound desc = could not find container \"a4e5361c26385a0c7615c7df337ebaa69e3c81c4732e2107b8af0b9b9a32ae83\": container with ID starting with a4e5361c26385a0c7615c7df337ebaa69e3c81c4732e2107b8af0b9b9a32ae83 not found: ID does not exist"
Jul 24 03:23:19.572020 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:19.571987    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" path="/var/lib/kubelet/pods/a43140e9-18ba-44e5-ba7a-03979b7613cc/volumes"
Jul 24 03:23:20.525227 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.525187    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 03:23:20.525536 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.525506    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="prometheus" containerID="cri-o://4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b" gracePeriod=600
Jul 24 03:23:20.525658 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.525526    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="kube-rbac-proxy" containerID="cri-o://c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c" gracePeriod=600
Jul 24 03:23:20.525658 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.525561    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="thanos-sidecar" containerID="cri-o://848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316" gracePeriod=600
Jul 24 03:23:20.525658 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.525609    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23" gracePeriod=600
Jul 24 03:23:20.525658 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.525569    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="kube-rbac-proxy-web" containerID="cri-o://54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68" gracePeriod=600
Jul 24 03:23:20.525971 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.525567    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="config-reloader" containerID="cri-o://45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e" gracePeriod=600
Jul 24 03:23:20.785074 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.784999    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:20.866289 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866260    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-kubelet-serving-ca-bundle\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866321    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1af0e672-5a9e-45f8-a3fd-44bc329168b7-config-out\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866338    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1af0e672-5a9e-45f8-a3fd-44bc329168b7-tls-assets\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866358    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-web-config\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866378    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-thanos-prometheus-http-client-file\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866401    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv6z5\" (UniqueName: \"kubernetes.io/projected/1af0e672-5a9e-45f8-a3fd-44bc329168b7-kube-api-access-nv6z5\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866508 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866489    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-0\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866822 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866533    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-1\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866822 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866569    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-grpc-tls\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866822 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866605    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-metrics-client-certs\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866822 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866629    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-2\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866822 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866662    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-serving-certs-ca-bundle\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866822 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866690    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866822 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866691    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:23:20.866822 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866725    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-metrics-client-ca\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866822 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866754    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-tls\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.866822 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866803    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-config\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.867489 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866833    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-trusted-ca-bundle\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.867489 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866861    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-kube-rbac-proxy\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.867489 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866910    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.867489 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.866934    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-db\") pod \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\" (UID: \"1af0e672-5a9e-45f8-a3fd-44bc329168b7\") "
Jul 24 03:23:20.867489 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.867215    2574 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.868251 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.868219    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:23:20.868519 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.868496    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:23:20.869119 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.868753    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:23:20.869119 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.869045    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1af0e672-5a9e-45f8-a3fd-44bc329168b7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:23:20.869803 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.869772    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1af0e672-5a9e-45f8-a3fd-44bc329168b7-kube-api-access-nv6z5" (OuterVolumeSpecName: "kube-api-access-nv6z5") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "kube-api-access-nv6z5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:23:20.869901 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.869814    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:23:20.869901 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.869865    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:23:20.870020 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.869983    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:20.870468 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.870419    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:23:20.870855 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.870817    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:20.870950 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.870921    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:23:20.872650 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.872623    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:20.872757 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.872740    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1af0e672-5a9e-45f8-a3fd-44bc329168b7-config-out" (OuterVolumeSpecName: "config-out") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:23:20.873337 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.873244    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:20.873337 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.873257    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:20.873337 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.873283    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:20.873658 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.873458    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-config" (OuterVolumeSpecName: "config") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:20.874113 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.874078    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:20.881897 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.881869    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-web-config" (OuterVolumeSpecName: "web-config") pod "1af0e672-5a9e-45f8-a3fd-44bc329168b7" (UID: "1af0e672-5a9e-45f8-a3fd-44bc329168b7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 03:23:20.967699 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967646    2574 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-metrics-client-certs\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967699 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967690    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967699 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967702    2574 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967699 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967713    2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967723    2574 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-configmap-metrics-client-ca\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967732    2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967741    2574 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-config\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967750    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967759    2574 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-kube-rbac-proxy\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967768    2574 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967778    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-db\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967785    2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1af0e672-5a9e-45f8-a3fd-44bc329168b7-config-out\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967793    2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1af0e672-5a9e-45f8-a3fd-44bc329168b7-tls-assets\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967800    2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-web-config\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967807    2574 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967816    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv6z5\" (UniqueName: \"kubernetes.io/projected/1af0e672-5a9e-45f8-a3fd-44bc329168b7-kube-api-access-nv6z5\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967824    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967832    2574 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1af0e672-5a9e-45f8-a3fd-44bc329168b7-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:20.967973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:20.967840    2574 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1af0e672-5a9e-45f8-a3fd-44bc329168b7-secret-grpc-tls\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:23:21.133261 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133167    2574 generic.go:356] "Generic (PLEG): container finished" podID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerID="2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23" exitCode=0
Jul 24 03:23:21.133261 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133196    2574 generic.go:356] "Generic (PLEG): container finished" podID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerID="c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c" exitCode=0
Jul 24 03:23:21.133261 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133204    2574 generic.go:356] "Generic (PLEG): container finished" podID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerID="54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68" exitCode=0
Jul 24 03:23:21.133261 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133210    2574 generic.go:356] "Generic (PLEG): container finished" podID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerID="848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316" exitCode=0
Jul 24 03:23:21.133261 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133217    2574 generic.go:356] "Generic (PLEG): container finished" podID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerID="45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e" exitCode=0
Jul 24 03:23:21.133261 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133221    2574 generic.go:356] "Generic (PLEG): container finished" podID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerID="4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b" exitCode=0
Jul 24 03:23:21.133261 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133245    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerDied","Data":"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"}
Jul 24 03:23:21.133749 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133273    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.133749 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133366    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerDied","Data":"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"}
Jul 24 03:23:21.133749 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133396    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerDied","Data":"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"}
Jul 24 03:23:21.133749 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133421    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerDied","Data":"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"}
Jul 24 03:23:21.133749 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133448    2574 scope.go:117] "RemoveContainer" containerID="2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"
Jul 24 03:23:21.133749 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133459    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerDied","Data":"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"}
Jul 24 03:23:21.133749 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133482    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerDied","Data":"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"}
Jul 24 03:23:21.133749 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.133505    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1af0e672-5a9e-45f8-a3fd-44bc329168b7","Type":"ContainerDied","Data":"54bec76a8e1a917b189ae44d8a928a07f3387104ad513d69f0c463bafcaa270c"}
Jul 24 03:23:21.141914 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.141883    2574 scope.go:117] "RemoveContainer" containerID="c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"
Jul 24 03:23:21.149462 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.149445    2574 scope.go:117] "RemoveContainer" containerID="54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"
Jul 24 03:23:21.157772 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.157743    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 03:23:21.158045 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.158025    2574 scope.go:117] "RemoveContainer" containerID="848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"
Jul 24 03:23:21.161510 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.161486    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 03:23:21.165806 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.165788    2574 scope.go:117] "RemoveContainer" containerID="45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"
Jul 24 03:23:21.173120 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.173096    2574 scope.go:117] "RemoveContainer" containerID="4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"
Jul 24 03:23:21.181671 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.181651    2574 scope.go:117] "RemoveContainer" containerID="ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"
Jul 24 03:23:21.187782 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.187758    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 03:23:21.188107 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188091    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="config-reloader"
Jul 24 03:23:21.188180 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188111    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="config-reloader"
Jul 24 03:23:21.188180 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188125    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="kube-rbac-proxy"
Jul 24 03:23:21.188180 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188133    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="kube-rbac-proxy"
Jul 24 03:23:21.188180 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188145    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="kube-rbac-proxy-thanos"
Jul 24 03:23:21.188180 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188153    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="kube-rbac-proxy-thanos"
Jul 24 03:23:21.188180 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188164    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="kube-rbac-proxy-metric"
Jul 24 03:23:21.188180 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188174    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="kube-rbac-proxy-metric"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188194    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="init-config-reloader"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188203    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="init-config-reloader"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188212    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="prometheus"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188219    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="prometheus"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188229    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="kube-rbac-proxy"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188236    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="kube-rbac-proxy"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188250    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="alertmanager"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188260    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="alertmanager"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188272    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="prom-label-proxy"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188280    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="prom-label-proxy"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188293    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="kube-rbac-proxy-web"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188302    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="kube-rbac-proxy-web"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188311    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="thanos-sidecar"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188319    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="thanos-sidecar"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188327    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="kube-rbac-proxy-web"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188335    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="kube-rbac-proxy-web"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188348    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="init-config-reloader"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188356    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="init-config-reloader"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188370    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="config-reloader"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188378    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="config-reloader"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188499    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="kube-rbac-proxy"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188511    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="kube-rbac-proxy-metric"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188521    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="thanos-sidecar"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188533    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="prometheus"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188543    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="config-reloader"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188552    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="prom-label-proxy"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188559    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="kube-rbac-proxy"
Jul 24 03:23:21.188556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188568    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="kube-rbac-proxy-thanos"
Jul 24 03:23:21.189844 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188576    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="kube-rbac-proxy-web"
Jul 24 03:23:21.189844 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188585    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="config-reloader"
Jul 24 03:23:21.189844 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188594    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a43140e9-18ba-44e5-ba7a-03979b7613cc" containerName="alertmanager"
Jul 24 03:23:21.189844 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.188602    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" containerName="kube-rbac-proxy-web"
Jul 24 03:23:21.190126 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.189945    2574 scope.go:117] "RemoveContainer" containerID="2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"
Jul 24 03:23:21.190281 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:21.190260    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23\": container with ID starting with 2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23 not found: ID does not exist" containerID="2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"
Jul 24 03:23:21.190341 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.190288    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"} err="failed to get container status \"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23\": rpc error: code = NotFound desc = could not find container \"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23\": container with ID starting with 2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23 not found: ID does not exist"
Jul 24 03:23:21.190341 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.190308    2574 scope.go:117] "RemoveContainer" containerID="c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"
Jul 24 03:23:21.190617 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:21.190597    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c\": container with ID starting with c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c not found: ID does not exist" containerID="c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"
Jul 24 03:23:21.190682 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.190634    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"} err="failed to get container status \"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c\": rpc error: code = NotFound desc = could not find container \"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c\": container with ID starting with c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c not found: ID does not exist"
Jul 24 03:23:21.190682 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.190653    2574 scope.go:117] "RemoveContainer" containerID="54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"
Jul 24 03:23:21.190918 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:21.190877    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68\": container with ID starting with 54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68 not found: ID does not exist" containerID="54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"
Jul 24 03:23:21.190918 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.190903    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"} err="failed to get container status \"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68\": rpc error: code = NotFound desc = could not find container \"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68\": container with ID starting with 54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68 not found: ID does not exist"
Jul 24 03:23:21.190988 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.190924    2574 scope.go:117] "RemoveContainer" containerID="848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"
Jul 24 03:23:21.191172 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:21.191152    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316\": container with ID starting with 848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316 not found: ID does not exist" containerID="848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"
Jul 24 03:23:21.191259 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.191175    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"} err="failed to get container status \"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316\": rpc error: code = NotFound desc = could not find container \"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316\": container with ID starting with 848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316 not found: ID does not exist"
Jul 24 03:23:21.191259 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.191189    2574 scope.go:117] "RemoveContainer" containerID="45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"
Jul 24 03:23:21.191415 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:21.191400    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e\": container with ID starting with 45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e not found: ID does not exist" containerID="45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"
Jul 24 03:23:21.191491 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.191417    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"} err="failed to get container status \"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e\": rpc error: code = NotFound desc = could not find container \"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e\": container with ID starting with 45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e not found: ID does not exist"
Jul 24 03:23:21.191491 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.191462    2574 scope.go:117] "RemoveContainer" containerID="4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"
Jul 24 03:23:21.191702 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:21.191687    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b\": container with ID starting with 4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b not found: ID does not exist" containerID="4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"
Jul 24 03:23:21.191764 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.191706    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"} err="failed to get container status \"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b\": rpc error: code = NotFound desc = could not find container \"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b\": container with ID starting with 4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b not found: ID does not exist"
Jul 24 03:23:21.191764 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.191721    2574 scope.go:117] "RemoveContainer" containerID="ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"
Jul 24 03:23:21.191950 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:23:21.191934    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2\": container with ID starting with ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2 not found: ID does not exist" containerID="ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"
Jul 24 03:23:21.191988 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.191955    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"} err="failed to get container status \"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2\": rpc error: code = NotFound desc = could not find container \"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2\": container with ID starting with ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2 not found: ID does not exist"
Jul 24 03:23:21.191988 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.191968    2574 scope.go:117] "RemoveContainer" containerID="2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"
Jul 24 03:23:21.192188 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.192166    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"} err="failed to get container status \"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23\": rpc error: code = NotFound desc = could not find container \"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23\": container with ID starting with 2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23 not found: ID does not exist"
Jul 24 03:23:21.192188 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.192188    2574 scope.go:117] "RemoveContainer" containerID="c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"
Jul 24 03:23:21.192558 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.192362    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"} err="failed to get container status \"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c\": rpc error: code = NotFound desc = could not find container \"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c\": container with ID starting with c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c not found: ID does not exist"
Jul 24 03:23:21.192558 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.192376    2574 scope.go:117] "RemoveContainer" containerID="54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"
Jul 24 03:23:21.192696 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.192666    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"} err="failed to get container status \"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68\": rpc error: code = NotFound desc = could not find container \"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68\": container with ID starting with 54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68 not found: ID does not exist"
Jul 24 03:23:21.192696 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.192692    2574 scope.go:117] "RemoveContainer" containerID="848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"
Jul 24 03:23:21.192989 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.192946    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"} err="failed to get container status \"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316\": rpc error: code = NotFound desc = could not find container \"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316\": container with ID starting with 848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316 not found: ID does not exist"
Jul 24 03:23:21.192989 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.192990    2574 scope.go:117] "RemoveContainer" containerID="45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"
Jul 24 03:23:21.193142 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.193110    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.193783 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.193327    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"} err="failed to get container status \"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e\": rpc error: code = NotFound desc = could not find container \"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e\": container with ID starting with 45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e not found: ID does not exist"
Jul 24 03:23:21.193783 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.193352    2574 scope.go:117] "RemoveContainer" containerID="4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"
Jul 24 03:23:21.194277 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.194147    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"} err="failed to get container status \"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b\": rpc error: code = NotFound desc = could not find container \"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b\": container with ID starting with 4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b not found: ID does not exist"
Jul 24 03:23:21.194277 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.194181    2574 scope.go:117] "RemoveContainer" containerID="ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"
Jul 24 03:23:21.194676 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.194648    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"} err="failed to get container status \"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2\": rpc error: code = NotFound desc = could not find container \"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2\": container with ID starting with ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2 not found: ID does not exist"
Jul 24 03:23:21.194795 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.194775    2574 scope.go:117] "RemoveContainer" containerID="2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"
Jul 24 03:23:21.195943 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.195595    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"} err="failed to get container status \"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23\": rpc error: code = NotFound desc = could not find container \"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23\": container with ID starting with 2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23 not found: ID does not exist"
Jul 24 03:23:21.195943 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.195621    2574 scope.go:117] "RemoveContainer" containerID="c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"
Jul 24 03:23:21.196205 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.196157    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 03:23:21.196298 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.196206    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 03:23:21.196298 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.196213    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 03:23:21.196515 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.196498    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 03:23:21.196796 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.196644    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k8vn5\""
Jul 24 03:23:21.196796 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.196683    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"} err="failed to get container status \"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c\": rpc error: code = NotFound desc = could not find container \"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c\": container with ID starting with c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c not found: ID does not exist"
Jul 24 03:23:21.196796 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.196708    2574 scope.go:117] "RemoveContainer" containerID="54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"
Jul 24 03:23:21.196796 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.196747    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 03:23:21.197136 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.196902    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 03:23:21.197136 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.196999    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 03:23:21.197136 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.197053    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 03:23:21.197136 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.197082    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 03:23:21.197375 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.197357    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 03:23:21.197611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.197596    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 03:23:21.198226 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.197888    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"} err="failed to get container status \"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68\": rpc error: code = NotFound desc = could not find container \"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68\": container with ID starting with 54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68 not found: ID does not exist"
Jul 24 03:23:21.198226 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.197914    2574 scope.go:117] "RemoveContainer" containerID="848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"
Jul 24 03:23:21.199965 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.199946    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 03:23:21.200202 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.199954    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8ubppmkdidbep\""
Jul 24 03:23:21.201006 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.200478    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"} err="failed to get container status \"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316\": rpc error: code = NotFound desc = could not find container \"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316\": container with ID starting with 848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316 not found: ID does not exist"
Jul 24 03:23:21.201006 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.200508    2574 scope.go:117] "RemoveContainer" containerID="45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"
Jul 24 03:23:21.201006 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.200882    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"} err="failed to get container status \"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e\": rpc error: code = NotFound desc = could not find container \"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e\": container with ID starting with 45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e not found: ID does not exist"
Jul 24 03:23:21.201006 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.200905    2574 scope.go:117] "RemoveContainer" containerID="4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"
Jul 24 03:23:21.201582 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.201561    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"} err="failed to get container status \"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b\": rpc error: code = NotFound desc = could not find container \"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b\": container with ID starting with 4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b not found: ID does not exist"
Jul 24 03:23:21.201691 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.201584    2574 scope.go:117] "RemoveContainer" containerID="ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"
Jul 24 03:23:21.201854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.201835    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"} err="failed to get container status \"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2\": rpc error: code = NotFound desc = could not find container \"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2\": container with ID starting with ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2 not found: ID does not exist"
Jul 24 03:23:21.201940 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.201856    2574 scope.go:117] "RemoveContainer" containerID="2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"
Jul 24 03:23:21.202157 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.202135    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"} err="failed to get container status \"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23\": rpc error: code = NotFound desc = could not find container \"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23\": container with ID starting with 2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23 not found: ID does not exist"
Jul 24 03:23:21.202228 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.202160    2574 scope.go:117] "RemoveContainer" containerID="c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"
Jul 24 03:23:21.202294 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.202242    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 03:23:21.202499 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.202445    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"} err="failed to get container status \"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c\": rpc error: code = NotFound desc = could not find container \"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c\": container with ID starting with c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c not found: ID does not exist"
Jul 24 03:23:21.202499 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.202480    2574 scope.go:117] "RemoveContainer" containerID="54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"
Jul 24 03:23:21.202827 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.202804    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"} err="failed to get container status \"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68\": rpc error: code = NotFound desc = could not find container \"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68\": container with ID starting with 54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68 not found: ID does not exist"
Jul 24 03:23:21.202902 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.202822    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 03:23:21.202902 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.202830    2574 scope.go:117] "RemoveContainer" containerID="848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"
Jul 24 03:23:21.203096 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.203065    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"} err="failed to get container status \"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316\": rpc error: code = NotFound desc = could not find container \"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316\": container with ID starting with 848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316 not found: ID does not exist"
Jul 24 03:23:21.203096 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.203090    2574 scope.go:117] "RemoveContainer" containerID="45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"
Jul 24 03:23:21.203371 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.203338    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"} err="failed to get container status \"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e\": rpc error: code = NotFound desc = could not find container \"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e\": container with ID starting with 45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e not found: ID does not exist"
Jul 24 03:23:21.203448 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.203373    2574 scope.go:117] "RemoveContainer" containerID="4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"
Jul 24 03:23:21.203622 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.203599    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"} err="failed to get container status \"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b\": rpc error: code = NotFound desc = could not find container \"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b\": container with ID starting with 4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b not found: ID does not exist"
Jul 24 03:23:21.203668 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.203624    2574 scope.go:117] "RemoveContainer" containerID="ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"
Jul 24 03:23:21.203883 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.203861    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"} err="failed to get container status \"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2\": rpc error: code = NotFound desc = could not find container \"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2\": container with ID starting with ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2 not found: ID does not exist"
Jul 24 03:23:21.204086 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.203881    2574 scope.go:117] "RemoveContainer" containerID="2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"
Jul 24 03:23:21.204280 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.204256    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"} err="failed to get container status \"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23\": rpc error: code = NotFound desc = could not find container \"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23\": container with ID starting with 2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23 not found: ID does not exist"
Jul 24 03:23:21.204352 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.204280    2574 scope.go:117] "RemoveContainer" containerID="c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"
Jul 24 03:23:21.204589 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.204564    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"} err="failed to get container status \"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c\": rpc error: code = NotFound desc = could not find container \"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c\": container with ID starting with c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c not found: ID does not exist"
Jul 24 03:23:21.204683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.204593    2574 scope.go:117] "RemoveContainer" containerID="54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"
Jul 24 03:23:21.204899 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.204875    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"} err="failed to get container status \"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68\": rpc error: code = NotFound desc = could not find container \"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68\": container with ID starting with 54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68 not found: ID does not exist"
Jul 24 03:23:21.204999 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.204902    2574 scope.go:117] "RemoveContainer" containerID="848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"
Jul 24 03:23:21.205230 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.205206    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"} err="failed to get container status \"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316\": rpc error: code = NotFound desc = could not find container \"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316\": container with ID starting with 848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316 not found: ID does not exist"
Jul 24 03:23:21.205334 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.205233    2574 scope.go:117] "RemoveContainer" containerID="45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"
Jul 24 03:23:21.207007 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.206977    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"} err="failed to get container status \"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e\": rpc error: code = NotFound desc = could not find container \"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e\": container with ID starting with 45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e not found: ID does not exist"
Jul 24 03:23:21.207118 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.207008    2574 scope.go:117] "RemoveContainer" containerID="4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"
Jul 24 03:23:21.207301 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.207278    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"} err="failed to get container status \"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b\": rpc error: code = NotFound desc = could not find container \"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b\": container with ID starting with 4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b not found: ID does not exist"
Jul 24 03:23:21.207393 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.207304    2574 scope.go:117] "RemoveContainer" containerID="ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"
Jul 24 03:23:21.207616 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.207589    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"} err="failed to get container status \"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2\": rpc error: code = NotFound desc = could not find container \"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2\": container with ID starting with ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2 not found: ID does not exist"
Jul 24 03:23:21.207616 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.207617    2574 scope.go:117] "RemoveContainer" containerID="2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"
Jul 24 03:23:21.207888 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.207866    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23"} err="failed to get container status \"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23\": rpc error: code = NotFound desc = could not find container \"2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23\": container with ID starting with 2a3ce3c88bb57eaf57fd6024df736cb67f47965f25f684e39eb8cb9559588a23 not found: ID does not exist"
Jul 24 03:23:21.207941 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.207891    2574 scope.go:117] "RemoveContainer" containerID="c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"
Jul 24 03:23:21.208084 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.208065    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 03:23:21.208170 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.208123    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c"} err="failed to get container status \"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c\": rpc error: code = NotFound desc = could not find container \"c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c\": container with ID starting with c6abaf09466453dc2dedcf5a97da7ca4293446413ebad17ef60df887aae76d1c not found: ID does not exist"
Jul 24 03:23:21.208170 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.208147    2574 scope.go:117] "RemoveContainer" containerID="54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"
Jul 24 03:23:21.208416 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.208399    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68"} err="failed to get container status \"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68\": rpc error: code = NotFound desc = could not find container \"54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68\": container with ID starting with 54a18c0ef1b2dc3f41d5c67baae50ecec0a6d243172d800653ef5f236040fa68 not found: ID does not exist"
Jul 24 03:23:21.208506 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.208417    2574 scope.go:117] "RemoveContainer" containerID="848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"
Jul 24 03:23:21.208679 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.208663    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316"} err="failed to get container status \"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316\": rpc error: code = NotFound desc = could not find container \"848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316\": container with ID starting with 848e73963ae25d382d4a19133c3a990611b94c12db9c0475a6773296c46e8316 not found: ID does not exist"
Jul 24 03:23:21.208725 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.208680    2574 scope.go:117] "RemoveContainer" containerID="45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"
Jul 24 03:23:21.208874 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.208854    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e"} err="failed to get container status \"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e\": rpc error: code = NotFound desc = could not find container \"45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e\": container with ID starting with 45631265a72529ecfa3c85e09c216b05faa1d924ec55c198570ccd4fc2549b4e not found: ID does not exist"
Jul 24 03:23:21.208942 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.208875    2574 scope.go:117] "RemoveContainer" containerID="4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"
Jul 24 03:23:21.209079 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.209060    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b"} err="failed to get container status \"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b\": rpc error: code = NotFound desc = could not find container \"4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b\": container with ID starting with 4509dddf84ab056d81ba28505879ccff131df09a3a2bde923c1a0b76e2a08f3b not found: ID does not exist"
Jul 24 03:23:21.209079 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.209078    2574 scope.go:117] "RemoveContainer" containerID="ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"
Jul 24 03:23:21.209345 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.209322    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2"} err="failed to get container status \"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2\": rpc error: code = NotFound desc = could not find container \"ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2\": container with ID starting with ca3b2dbc6e964b1f9895230150840420383dbf508f90c4258893a284a5c777c2 not found: ID does not exist"
Jul 24 03:23:21.270274 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270226    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270274 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270274    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270557 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270334    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270557 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270374    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-config-out\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270557 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270391    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270557 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270411    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270557 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270488    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270557 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270508    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-web-config\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270557 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270531    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-config\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270814 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270560    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270814 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270583    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270814 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270623    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270814 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270665    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270814 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270680    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270814 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270706    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270814 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270726    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270814 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270757    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.270814 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270798    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpvpx\" (UniqueName: \"kubernetes.io/projected/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-kube-api-access-xpvpx\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.271081 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270830    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.271081 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.270850    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372134 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372097    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372271 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372154    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372271 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372271 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372194    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372271 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372209    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372271 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372244    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372271 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372268    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpvpx\" (UniqueName: \"kubernetes.io/projected/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-kube-api-access-xpvpx\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372620 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372297    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372620 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372572    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372714 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372714 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372651    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372714 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372692    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372876 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372721    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-config-out\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372876 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372746    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372876 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372779    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372876 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372812    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372876 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372843    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-web-config\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.372876 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372874    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-config\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.373333 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372902    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.373333 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.372929    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.373898 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.373570    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.373898 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.373610    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.374067 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.373932    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.376531 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.375797    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.376531 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.376214    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.376531 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.376325    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.376531 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.376409    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.376531 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.376451    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-config\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.376866 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.376587    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.377368 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.377025    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.377368 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.377051    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.377368 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.377293    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.377368 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.377344    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-config-out\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.377620 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.377375    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.377620 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.377387    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.377620 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.377513    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.377797 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.377776    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-web-config\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.378857 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.378836    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.379058 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.379040    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.379737 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.379721    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpvpx\" (UniqueName: \"kubernetes.io/projected/b0d695c4-fc68-4e09-b78e-82fc7b59abb9-kube-api-access-xpvpx\") pod \"prometheus-k8s-0\" (UID: \"b0d695c4-fc68-4e09-b78e-82fc7b59abb9\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.508136 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.508086    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:23:21.574537 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.574505    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1af0e672-5a9e-45f8-a3fd-44bc329168b7" path="/var/lib/kubelet/pods/1af0e672-5a9e-45f8-a3fd-44bc329168b7/volumes"
Jul 24 03:23:21.664096 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:21.663848    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 03:23:21.666518 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:23:21.666491    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0d695c4_fc68_4e09_b78e_82fc7b59abb9.slice/crio-64ea962188b5b21ab99efacbbd1a3fece47efff2328e5e57d15ba60052fe03ac WatchSource:0}: Error finding container 64ea962188b5b21ab99efacbbd1a3fece47efff2328e5e57d15ba60052fe03ac: Status 404 returned error can't find the container with id 64ea962188b5b21ab99efacbbd1a3fece47efff2328e5e57d15ba60052fe03ac
Jul 24 03:23:22.141252 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:22.141220    2574 generic.go:356] "Generic (PLEG): container finished" podID="b0d695c4-fc68-4e09-b78e-82fc7b59abb9" containerID="ed8ed8c576d6135e61834f0c52a1e4ef9e20783018c70c1ed7cf996f80699dc2" exitCode=0
Jul 24 03:23:22.141651 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:22.141311    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0d695c4-fc68-4e09-b78e-82fc7b59abb9","Type":"ContainerDied","Data":"ed8ed8c576d6135e61834f0c52a1e4ef9e20783018c70c1ed7cf996f80699dc2"}
Jul 24 03:23:22.141651 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:22.141383    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0d695c4-fc68-4e09-b78e-82fc7b59abb9","Type":"ContainerStarted","Data":"64ea962188b5b21ab99efacbbd1a3fece47efff2328e5e57d15ba60052fe03ac"}
Jul 24 03:23:23.149180 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:23.149139    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0d695c4-fc68-4e09-b78e-82fc7b59abb9","Type":"ContainerStarted","Data":"7a1e4afb74d0b2069658371403fc525cc3093042f89ca3591ae3840384f48c99"}
Jul 24 03:23:23.149745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:23.149264    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0d695c4-fc68-4e09-b78e-82fc7b59abb9","Type":"ContainerStarted","Data":"55fe5f961ba2e6a6d0b48baef09b8ec7600771daeccd5413a5dfeabe6ea35b13"}
Jul 24 03:23:23.149745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:23.149309    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0d695c4-fc68-4e09-b78e-82fc7b59abb9","Type":"ContainerStarted","Data":"ea1e044d8777245d048c2c238581700ace48573591a5f104af60ca27ad4223fc"}
Jul 24 03:23:23.149745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:23.149357    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0d695c4-fc68-4e09-b78e-82fc7b59abb9","Type":"ContainerStarted","Data":"0ee7cdca0f627f1b5a82e1267c6a15965f647bdfaa99b8eac424369a29c2c138"}
Jul 24 03:23:23.149745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:23.149392    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0d695c4-fc68-4e09-b78e-82fc7b59abb9","Type":"ContainerStarted","Data":"b28b30b66f23e27f63081dc87e59eae6d578cb7ec4003ea7ca11451676fe703c"}
Jul 24 03:23:23.149745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:23.149447    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0d695c4-fc68-4e09-b78e-82fc7b59abb9","Type":"ContainerStarted","Data":"091257af702c6649347ce7ea4bf682447164202810af69b4fe8c7d362d78bd2c"}
Jul 24 03:23:23.174577 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:23.174514    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.156122483 podStartE2EDuration="2.174491212s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.368729ms" isStatefulPod=false podCreationTimestamp="2026-07-24 03:23:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:23:23.172976913 +0000 UTC m=+358.136070200" watchObservedRunningTime="2026-07-24 03:23:23.174491212 +0000 UTC m=+358.137584479"
Jul 24 03:23:26.508837 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:23:26.508801    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:24:21.508650 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:24:21.508553    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:24:21.525123 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:24:21.525093    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:24:22.409319 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:24:22.409290    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 03:26:11.026387 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.026349    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"]
Jul 24 03:26:11.028685 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.028670    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.033125 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.033100    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 03:26:11.033242 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.033188    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 03:26:11.033869 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.033856    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 03:26:11.034397 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.034376    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q7bbg\""
Jul 24 03:26:11.034397 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.034391    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 03:26:11.049518 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.049483    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qx7p\" (UniqueName: \"kubernetes.io/projected/68a31406-ae8d-4a3a-b9b4-3740e31aecee-kube-api-access-8qx7p\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.049661 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.049527    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68a31406-ae8d-4a3a-b9b4-3740e31aecee-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.049661 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.049552    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a31406-ae8d-4a3a-b9b4-3740e31aecee-tmp\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.049661 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.049574    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/68a31406-ae8d-4a3a-b9b4-3740e31aecee-manifests\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.049661 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.049637    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/68a31406-ae8d-4a3a-b9b4-3740e31aecee-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.058368 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.058331    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"]
Jul 24 03:26:11.150544 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.150491    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qx7p\" (UniqueName: \"kubernetes.io/projected/68a31406-ae8d-4a3a-b9b4-3740e31aecee-kube-api-access-8qx7p\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.150753 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.150555    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68a31406-ae8d-4a3a-b9b4-3740e31aecee-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.150753 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.150584    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a31406-ae8d-4a3a-b9b4-3740e31aecee-tmp\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.150753 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.150611    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/68a31406-ae8d-4a3a-b9b4-3740e31aecee-manifests\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.150753 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.150643    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/68a31406-ae8d-4a3a-b9b4-3740e31aecee-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.151045 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.151025    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/68a31406-ae8d-4a3a-b9b4-3740e31aecee-manifests\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.153126 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.153096    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a31406-ae8d-4a3a-b9b4-3740e31aecee-tmp\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.153237 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.153201    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/68a31406-ae8d-4a3a-b9b4-3740e31aecee-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.153237 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.153218    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68a31406-ae8d-4a3a-b9b4-3740e31aecee-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.170587 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.170555    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qx7p\" (UniqueName: \"kubernetes.io/projected/68a31406-ae8d-4a3a-b9b4-3740e31aecee-kube-api-access-8qx7p\") pod \"opendatahub-operator-controller-manager-67f4587fcc-6zg4n\" (UID: \"68a31406-ae8d-4a3a-b9b4-3740e31aecee\") " pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.339724 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.339638    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:11.484741 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.484713    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"]
Jul 24 03:26:11.490821 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:26:11.490793    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a31406_ae8d_4a3a_b9b4_3740e31aecee.slice/crio-36646527dc35da3cbef41dc5d10fa98e6d8908451a60216b68fbc306558a4a8a WatchSource:0}: Error finding container 36646527dc35da3cbef41dc5d10fa98e6d8908451a60216b68fbc306558a4a8a: Status 404 returned error can't find the container with id 36646527dc35da3cbef41dc5d10fa98e6d8908451a60216b68fbc306558a4a8a
Jul 24 03:26:11.492647 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.492626    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 03:26:11.834235 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:11.834204    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n" event={"ID":"68a31406-ae8d-4a3a-b9b4-3740e31aecee","Type":"ContainerStarted","Data":"36646527dc35da3cbef41dc5d10fa98e6d8908451a60216b68fbc306558a4a8a"}
Jul 24 03:26:14.852351 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:14.852269    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n" event={"ID":"68a31406-ae8d-4a3a-b9b4-3740e31aecee","Type":"ContainerStarted","Data":"39c0ba19546dab86e5132dbba9bdad23b3558e23a45435fa1f9d85391507f4b6"}
Jul 24 03:26:15.859013 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:15.858922    2574 generic.go:356] "Generic (PLEG): container finished" podID="68a31406-ae8d-4a3a-b9b4-3740e31aecee" containerID="39c0ba19546dab86e5132dbba9bdad23b3558e23a45435fa1f9d85391507f4b6" exitCode=0
Jul 24 03:26:15.859377 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:15.859011    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n" event={"ID":"68a31406-ae8d-4a3a-b9b4-3740e31aecee","Type":"ContainerDied","Data":"39c0ba19546dab86e5132dbba9bdad23b3558e23a45435fa1f9d85391507f4b6"}
Jul 24 03:26:16.865366 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:16.865331    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n" event={"ID":"68a31406-ae8d-4a3a-b9b4-3740e31aecee","Type":"ContainerStarted","Data":"4ab267e8f09b3bd74c57d4f335140070e979f28c46fe7503e1229f15cf6557e2"}
Jul 24 03:26:16.865811 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:16.865474    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:16.895622 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:16.895570    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n" podStartSLOduration=2.493658389 podStartE2EDuration="6.895551236s" totalImagesPullingTime="3.397583724s" totalInitContainerRuntime="1.004309123s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:26:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:26:16.894028617 +0000 UTC m=+531.857121914" watchObservedRunningTime="2026-07-24 03:26:16.895551236 +0000 UTC m=+531.858644523"
Jul 24 03:26:27.872517 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:27.872487    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67f4587fcc-6zg4n"
Jul 24 03:26:41.157839 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.157805    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"]
Jul 24 03:26:41.167848 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.167821    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.171837 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.171806    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 03:26:41.172005 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.171890    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 24 03:26:41.172080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.172067    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 24 03:26:41.173125 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.173109    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 24 03:26:41.175560 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.175538    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-s5gzl\""
Jul 24 03:26:41.176862 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.176843    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"]
Jul 24 03:26:41.331168 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.331128    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aedf099-d724-413b-a2de-d960cc12170e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.331168 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.331169    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4aedf099-d724-413b-a2de-d960cc12170e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.331487 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.331252    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgvxv\" (UniqueName: \"kubernetes.io/projected/4aedf099-d724-413b-a2de-d960cc12170e-kube-api-access-kgvxv\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.331487 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.331310    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4aedf099-d724-413b-a2de-d960cc12170e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.331487 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.331336    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aedf099-d724-413b-a2de-d960cc12170e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.331487 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.331374    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4aedf099-d724-413b-a2de-d960cc12170e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.331631 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.331519    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4aedf099-d724-413b-a2de-d960cc12170e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.432719 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.432633    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4aedf099-d724-413b-a2de-d960cc12170e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.432719 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.432687    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4aedf099-d724-413b-a2de-d960cc12170e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.432719 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.432715    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aedf099-d724-413b-a2de-d960cc12170e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.432978 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.432734    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4aedf099-d724-413b-a2de-d960cc12170e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.432978 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.432776    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgvxv\" (UniqueName: \"kubernetes.io/projected/4aedf099-d724-413b-a2de-d960cc12170e-kube-api-access-kgvxv\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.432978 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.432815    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4aedf099-d724-413b-a2de-d960cc12170e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.432978 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.432844    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aedf099-d724-413b-a2de-d960cc12170e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.433570 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.433515    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4aedf099-d724-413b-a2de-d960cc12170e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.435304 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.435274    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4aedf099-d724-413b-a2de-d960cc12170e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.435399 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.435345    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4aedf099-d724-413b-a2de-d960cc12170e-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.435489 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.435459    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4aedf099-d724-413b-a2de-d960cc12170e-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.435544 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.435526    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aedf099-d724-413b-a2de-d960cc12170e-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.452501 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.452470    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aedf099-d724-413b-a2de-d960cc12170e-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.452501 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.452494    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgvxv\" (UniqueName: \"kubernetes.io/projected/4aedf099-d724-413b-a2de-d960cc12170e-kube-api-access-kgvxv\") pod \"istiod-openshift-gateway-5677ccb49b-56l8s\" (UID: \"4aedf099-d724-413b-a2de-d960cc12170e\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.479129 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.479095    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:41.657760 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.655915    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"]
Jul 24 03:26:41.659518 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:26:41.659486    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aedf099_d724_413b_a2de_d960cc12170e.slice/crio-00ab8fd8f49d7d03202a4a0a815c1de1c675638fab948e4cad35a194af94b0eb WatchSource:0}: Error finding container 00ab8fd8f49d7d03202a4a0a815c1de1c675638fab948e4cad35a194af94b0eb: Status 404 returned error can't find the container with id 00ab8fd8f49d7d03202a4a0a815c1de1c675638fab948e4cad35a194af94b0eb
Jul 24 03:26:41.977405 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:41.977372    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s" event={"ID":"4aedf099-d724-413b-a2de-d960cc12170e","Type":"ContainerStarted","Data":"00ab8fd8f49d7d03202a4a0a815c1de1c675638fab948e4cad35a194af94b0eb"}
Jul 24 03:26:42.699997 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.699889    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"]
Jul 24 03:26:42.707193 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.707166    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:42.714054 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.712939    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 03:26:42.714054 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.713212    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-47bb4\""
Jul 24 03:26:42.714054 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.713407    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 03:26:42.714054 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.713881    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 03:26:42.714054 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.713951    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 03:26:42.717651 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.717466    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 03:26:42.731589 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.731543    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"]
Jul 24 03:26:42.847887 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.847849    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8659a9e5-cf20-493c-9868-e123a8cba9b1-metrics-cert\") pod \"lws-controller-manager-54698f7c78-pjtd8\" (UID: \"8659a9e5-cf20-493c-9868-e123a8cba9b1\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:42.848056 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.847910    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x2k7\" (UniqueName: \"kubernetes.io/projected/8659a9e5-cf20-493c-9868-e123a8cba9b1-kube-api-access-2x2k7\") pod \"lws-controller-manager-54698f7c78-pjtd8\" (UID: \"8659a9e5-cf20-493c-9868-e123a8cba9b1\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:42.848056 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.848042    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8659a9e5-cf20-493c-9868-e123a8cba9b1-manager-config\") pod \"lws-controller-manager-54698f7c78-pjtd8\" (UID: \"8659a9e5-cf20-493c-9868-e123a8cba9b1\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:42.848183 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.848084    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8659a9e5-cf20-493c-9868-e123a8cba9b1-cert\") pod \"lws-controller-manager-54698f7c78-pjtd8\" (UID: \"8659a9e5-cf20-493c-9868-e123a8cba9b1\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:42.949495 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.949460    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8659a9e5-cf20-493c-9868-e123a8cba9b1-manager-config\") pod \"lws-controller-manager-54698f7c78-pjtd8\" (UID: \"8659a9e5-cf20-493c-9868-e123a8cba9b1\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:42.949674 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.949509    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8659a9e5-cf20-493c-9868-e123a8cba9b1-cert\") pod \"lws-controller-manager-54698f7c78-pjtd8\" (UID: \"8659a9e5-cf20-493c-9868-e123a8cba9b1\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:42.949674 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.949568    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8659a9e5-cf20-493c-9868-e123a8cba9b1-metrics-cert\") pod \"lws-controller-manager-54698f7c78-pjtd8\" (UID: \"8659a9e5-cf20-493c-9868-e123a8cba9b1\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:42.949674 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.949611    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x2k7\" (UniqueName: \"kubernetes.io/projected/8659a9e5-cf20-493c-9868-e123a8cba9b1-kube-api-access-2x2k7\") pod \"lws-controller-manager-54698f7c78-pjtd8\" (UID: \"8659a9e5-cf20-493c-9868-e123a8cba9b1\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:42.950305 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.950151    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8659a9e5-cf20-493c-9868-e123a8cba9b1-manager-config\") pod \"lws-controller-manager-54698f7c78-pjtd8\" (UID: \"8659a9e5-cf20-493c-9868-e123a8cba9b1\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:42.952317 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.952289    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8659a9e5-cf20-493c-9868-e123a8cba9b1-cert\") pod \"lws-controller-manager-54698f7c78-pjtd8\" (UID: \"8659a9e5-cf20-493c-9868-e123a8cba9b1\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:42.952526 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.952507    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8659a9e5-cf20-493c-9868-e123a8cba9b1-metrics-cert\") pod \"lws-controller-manager-54698f7c78-pjtd8\" (UID: \"8659a9e5-cf20-493c-9868-e123a8cba9b1\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:42.964739 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:42.964708    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x2k7\" (UniqueName: \"kubernetes.io/projected/8659a9e5-cf20-493c-9868-e123a8cba9b1-kube-api-access-2x2k7\") pod \"lws-controller-manager-54698f7c78-pjtd8\" (UID: \"8659a9e5-cf20-493c-9868-e123a8cba9b1\") " pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:43.023146 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:43.023113    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:43.202275 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:43.202245    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"]
Jul 24 03:26:43.203604 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:26:43.203583    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8659a9e5_cf20_493c_9868_e123a8cba9b1.slice/crio-b347ccce3dfb1f7d4720b9c7d6f06e66f075765b86f469a45f01a1c782f20e0f WatchSource:0}: Error finding container b347ccce3dfb1f7d4720b9c7d6f06e66f075765b86f469a45f01a1c782f20e0f: Status 404 returned error can't find the container with id b347ccce3dfb1f7d4720b9c7d6f06e66f075765b86f469a45f01a1c782f20e0f
Jul 24 03:26:43.991823 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:43.991784    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8" event={"ID":"8659a9e5-cf20-493c-9868-e123a8cba9b1","Type":"ContainerStarted","Data":"b347ccce3dfb1f7d4720b9c7d6f06e66f075765b86f469a45f01a1c782f20e0f"}
Jul 24 03:26:44.132666 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:44.132632    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 03:26:44.132735 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:44.132699    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 03:26:44.998592 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:44.998558    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s" event={"ID":"4aedf099-d724-413b-a2de-d960cc12170e","Type":"ContainerStarted","Data":"e4cd2b7546bc95f00c0c2d92658f08636d1bdbd2a3b2a17a6111a40a7453969e"}
Jul 24 03:26:44.999030 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:44.998828    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:45.082610 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:45.082541    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s" podStartSLOduration=1.61179158 podStartE2EDuration="4.082517729s" totalImagesPullingTime="2.470726149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:26:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:26:45.081299608 +0000 UTC m=+560.044392908" watchObservedRunningTime="2026-07-24 03:26:45.082517729 +0000 UTC m=+560.045611017"
Jul 24 03:26:46.006083 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:46.006036    2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-56l8s container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 03:26:46.006587 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:46.006104    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s" podUID="4aedf099-d724-413b-a2de-d960cc12170e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 03:26:47.009551 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:47.009514    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8" event={"ID":"8659a9e5-cf20-493c-9868-e123a8cba9b1","Type":"ContainerStarted","Data":"9f452d163c69692f23671a518f65e276f435a5fed375cdc171ff998f2cf03e0b"}
Jul 24 03:26:47.009938 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:47.009661    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:26:47.046881 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:47.046831    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8" podStartSLOduration=2.115371313 podStartE2EDuration="5.046813661s" totalImagesPullingTime="2.931442348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:26:47.046519274 +0000 UTC m=+562.009612562" watchObservedRunningTime="2026-07-24 03:26:47.046813661 +0000 UTC m=+562.009906947"
Jul 24 03:26:49.004969 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:49.004933    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-56l8s"
Jul 24 03:26:58.016628 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:26:58.016592    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-54698f7c78-pjtd8"
Jul 24 03:27:25.453191 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:27:25.453158    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:27:25.453802 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:27:25.453483    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:28:32.263842 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.263809    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"]
Jul 24 03:28:32.267219 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.267201    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"
Jul 24 03:28:32.269836 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.269810    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 03:28:32.270343 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.270325    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-44f58\""
Jul 24 03:28:32.270817 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.270804    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 03:28:32.282098 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.282073    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"]
Jul 24 03:28:32.358196 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.358157    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"]
Jul 24 03:28:32.358481 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:28:32.358448    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-njw4z], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz" podUID="0184d123-9fd6-4222-9bfa-9ec4e18093c0"
Jul 24 03:28:32.370364 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.370330    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"]
Jul 24 03:28:32.389798 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.389769    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"]
Jul 24 03:28:32.393391 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.393357    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"
Jul 24 03:28:32.412471 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.412414    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"]
Jul 24 03:28:32.432240 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.432207    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0184d123-9fd6-4222-9bfa-9ec4e18093c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hwrjz\" (UID: \"0184d123-9fd6-4222-9bfa-9ec4e18093c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"
Jul 24 03:28:32.432403 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.432272    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njw4z\" (UniqueName: \"kubernetes.io/projected/0184d123-9fd6-4222-9bfa-9ec4e18093c0-kube-api-access-njw4z\") pod \"kuadrant-operator-controller-manager-86878c769f-hwrjz\" (UID: \"0184d123-9fd6-4222-9bfa-9ec4e18093c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"
Jul 24 03:28:32.496480 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.496449    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"
Jul 24 03:28:32.499253 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:28:32.499221    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-hwrjz\" is forbidden: User \"system:node:ip-10-0-140-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-227.ec2.internal' and this object" podUID="0184d123-9fd6-4222-9bfa-9ec4e18093c0" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"
Jul 24 03:28:32.503040 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.503018    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"
Jul 24 03:28:32.505443 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:28:32.505402    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-hwrjz\" is forbidden: User \"system:node:ip-10-0-140-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-227.ec2.internal' and this object" podUID="0184d123-9fd6-4222-9bfa-9ec4e18093c0" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"
Jul 24 03:28:32.532926 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.532844    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njw4z\" (UniqueName: \"kubernetes.io/projected/0184d123-9fd6-4222-9bfa-9ec4e18093c0-kube-api-access-njw4z\") pod \"kuadrant-operator-controller-manager-86878c769f-hwrjz\" (UID: \"0184d123-9fd6-4222-9bfa-9ec4e18093c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"
Jul 24 03:28:32.532926 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.532889    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw95x\" (UniqueName: \"kubernetes.io/projected/df962657-4324-4884-ad07-012c86cdb09b-kube-api-access-dw95x\") pod \"kuadrant-operator-controller-manager-86878c769f-p6kfm\" (UID: \"df962657-4324-4884-ad07-012c86cdb09b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"
Jul 24 03:28:32.533129 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.532944    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df962657-4324-4884-ad07-012c86cdb09b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p6kfm\" (UID: \"df962657-4324-4884-ad07-012c86cdb09b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"
Jul 24 03:28:32.533129 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.532988    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0184d123-9fd6-4222-9bfa-9ec4e18093c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hwrjz\" (UID: \"0184d123-9fd6-4222-9bfa-9ec4e18093c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"
Jul 24 03:28:32.533299 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.533279    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0184d123-9fd6-4222-9bfa-9ec4e18093c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hwrjz\" (UID: \"0184d123-9fd6-4222-9bfa-9ec4e18093c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"
Jul 24 03:28:32.538221 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:28:32.538205    2574 projected.go:196] Error preparing data for projected volume kube-api-access-njw4z for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-hwrjz" not found
Jul 24 03:28:32.538308 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:28:32.538271    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0184d123-9fd6-4222-9bfa-9ec4e18093c0-kube-api-access-njw4z podName:0184d123-9fd6-4222-9bfa-9ec4e18093c0 nodeName:}" failed. No retries permitted until 2026-07-24 03:28:33.038253942 +0000 UTC m=+668.001347206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-njw4z" (UniqueName: "kubernetes.io/projected/0184d123-9fd6-4222-9bfa-9ec4e18093c0-kube-api-access-njw4z") pod "kuadrant-operator-controller-manager-86878c769f-hwrjz" (UID: "0184d123-9fd6-4222-9bfa-9ec4e18093c0") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-hwrjz" not found
Jul 24 03:28:32.633650 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.633614    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0184d123-9fd6-4222-9bfa-9ec4e18093c0-extensions-socket-volume\") pod \"0184d123-9fd6-4222-9bfa-9ec4e18093c0\" (UID: \"0184d123-9fd6-4222-9bfa-9ec4e18093c0\") "
Jul 24 03:28:32.633898 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.633873    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0184d123-9fd6-4222-9bfa-9ec4e18093c0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0184d123-9fd6-4222-9bfa-9ec4e18093c0" (UID: "0184d123-9fd6-4222-9bfa-9ec4e18093c0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:28:32.633980 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.633896    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw95x\" (UniqueName: \"kubernetes.io/projected/df962657-4324-4884-ad07-012c86cdb09b-kube-api-access-dw95x\") pod \"kuadrant-operator-controller-manager-86878c769f-p6kfm\" (UID: \"df962657-4324-4884-ad07-012c86cdb09b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"
Jul 24 03:28:32.634034 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.634008    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df962657-4324-4884-ad07-012c86cdb09b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p6kfm\" (UID: \"df962657-4324-4884-ad07-012c86cdb09b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"
Jul 24 03:28:32.634088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.634051    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njw4z\" (UniqueName: \"kubernetes.io/projected/0184d123-9fd6-4222-9bfa-9ec4e18093c0-kube-api-access-njw4z\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:28:32.634088 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.634067    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0184d123-9fd6-4222-9bfa-9ec4e18093c0-extensions-socket-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:28:32.634317 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.634300    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df962657-4324-4884-ad07-012c86cdb09b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-p6kfm\" (UID: \"df962657-4324-4884-ad07-012c86cdb09b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"
Jul 24 03:28:32.655482 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.655409    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw95x\" (UniqueName: \"kubernetes.io/projected/df962657-4324-4884-ad07-012c86cdb09b-kube-api-access-dw95x\") pod \"kuadrant-operator-controller-manager-86878c769f-p6kfm\" (UID: \"df962657-4324-4884-ad07-012c86cdb09b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"
Jul 24 03:28:32.705958 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.705918    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"
Jul 24 03:28:32.857962 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:32.857937    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"]
Jul 24 03:28:32.859790 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:28:32.859763    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf962657_4324_4884_ad07_012c86cdb09b.slice/crio-ff4b31f18dc14d4e17fee96c38de27ce1a0b41416d311f507a44c6a8bceeaaad WatchSource:0}: Error finding container ff4b31f18dc14d4e17fee96c38de27ce1a0b41416d311f507a44c6a8bceeaaad: Status 404 returned error can't find the container with id ff4b31f18dc14d4e17fee96c38de27ce1a0b41416d311f507a44c6a8bceeaaad
Jul 24 03:28:33.503623 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:33.503600    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"
Jul 24 03:28:33.503623 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:33.503620    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm" event={"ID":"df962657-4324-4884-ad07-012c86cdb09b","Type":"ContainerStarted","Data":"ff4b31f18dc14d4e17fee96c38de27ce1a0b41416d311f507a44c6a8bceeaaad"}
Jul 24 03:28:33.505778 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:28:33.505746    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-hwrjz\" is forbidden: User \"system:node:ip-10-0-140-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-227.ec2.internal' and this object" podUID="0184d123-9fd6-4222-9bfa-9ec4e18093c0" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"
Jul 24 03:28:33.512633 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:28:33.512603    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-hwrjz\" is forbidden: User \"system:node:ip-10-0-140-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-227.ec2.internal' and this object" podUID="0184d123-9fd6-4222-9bfa-9ec4e18093c0" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hwrjz"
Jul 24 03:28:33.575689 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:33.575645    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0184d123-9fd6-4222-9bfa-9ec4e18093c0" path="/var/lib/kubelet/pods/0184d123-9fd6-4222-9bfa-9ec4e18093c0/volumes"
Jul 24 03:28:37.527227 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:37.527190    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm" event={"ID":"df962657-4324-4884-ad07-012c86cdb09b","Type":"ContainerStarted","Data":"38b231a7a8352c00cf4fa4ee4505a31c695b476f048be46e1125dd688980cee9"}
Jul 24 03:28:37.527726 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:37.527321    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"
Jul 24 03:28:37.552020 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:37.551969    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm" podStartSLOduration=1.3992313809999999 podStartE2EDuration="5.5519522s" totalImagesPullingTime="4.152720819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:28:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:28:37.549237473 +0000 UTC m=+672.512330760" watchObservedRunningTime="2026-07-24 03:28:37.5519522 +0000 UTC m=+672.515045487"
Jul 24 03:28:48.534238 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:28:48.534197    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"
Jul 24 03:30:19.674359 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:19.674325    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-cz6z2"]
Jul 24 03:30:19.676736 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:19.676710    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-cz6z2"
Jul 24 03:30:19.680274 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:19.680252    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-nzcpz\""
Jul 24 03:30:19.680536 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:19.680523    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 24 03:30:19.699847 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:19.699802    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-cz6z2"]
Jul 24 03:30:19.831199 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:19.831165    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ea869f2-beb3-42bc-b932-d0df7d82515d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-cz6z2\" (UID: \"2ea869f2-beb3-42bc-b932-d0df7d82515d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cz6z2"
Jul 24 03:30:19.831388 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:19.831229    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdhxp\" (UniqueName: \"kubernetes.io/projected/2ea869f2-beb3-42bc-b932-d0df7d82515d-kube-api-access-rdhxp\") pod \"payload-processing-5cc6cd444b-cz6z2\" (UID: \"2ea869f2-beb3-42bc-b932-d0df7d82515d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cz6z2"
Jul 24 03:30:19.931970 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:19.931877    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ea869f2-beb3-42bc-b932-d0df7d82515d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-cz6z2\" (UID: \"2ea869f2-beb3-42bc-b932-d0df7d82515d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cz6z2"
Jul 24 03:30:19.931970 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:19.931932    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdhxp\" (UniqueName: \"kubernetes.io/projected/2ea869f2-beb3-42bc-b932-d0df7d82515d-kube-api-access-rdhxp\") pod \"payload-processing-5cc6cd444b-cz6z2\" (UID: \"2ea869f2-beb3-42bc-b932-d0df7d82515d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cz6z2"
Jul 24 03:30:19.932693 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:19.932667    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ea869f2-beb3-42bc-b932-d0df7d82515d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-cz6z2\" (UID: \"2ea869f2-beb3-42bc-b932-d0df7d82515d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cz6z2"
Jul 24 03:30:19.940085 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:19.940056    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdhxp\" (UniqueName: \"kubernetes.io/projected/2ea869f2-beb3-42bc-b932-d0df7d82515d-kube-api-access-rdhxp\") pod \"payload-processing-5cc6cd444b-cz6z2\" (UID: \"2ea869f2-beb3-42bc-b932-d0df7d82515d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cz6z2"
Jul 24 03:30:19.987228 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:19.987194    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-cz6z2"
Jul 24 03:30:20.131180 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:20.131145    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-cz6z2"]
Jul 24 03:30:20.134021 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:30:20.133996    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ea869f2_beb3_42bc_b932_d0df7d82515d.slice/crio-a2f9abd41461dc5c588881c2fa1d2da3e92c09bceba2cbc96e5e5899d3d1819a WatchSource:0}: Error finding container a2f9abd41461dc5c588881c2fa1d2da3e92c09bceba2cbc96e5e5899d3d1819a: Status 404 returned error can't find the container with id a2f9abd41461dc5c588881c2fa1d2da3e92c09bceba2cbc96e5e5899d3d1819a
Jul 24 03:30:21.010876 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:21.010825    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-cz6z2" event={"ID":"2ea869f2-beb3-42bc-b932-d0df7d82515d","Type":"ContainerStarted","Data":"a2f9abd41461dc5c588881c2fa1d2da3e92c09bceba2cbc96e5e5899d3d1819a"}
Jul 24 03:30:23.022894 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:23.022854    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-cz6z2" event={"ID":"2ea869f2-beb3-42bc-b932-d0df7d82515d","Type":"ContainerStarted","Data":"c48d6037191f416dc8018facf41d9e1e8cd9b60bf41e5158f9d619fac27d1a74"}
Jul 24 03:30:23.023279 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:23.022946    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-cz6z2"
Jul 24 03:30:23.038329 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:23.038276    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-cz6z2" podStartSLOduration=1.851164196 podStartE2EDuration="4.038260645s" totalImagesPullingTime="2.187096449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:30:23.037322444 +0000 UTC m=+778.000415746" watchObservedRunningTime="2026-07-24 03:30:23.038260645 +0000 UTC m=+778.001353936"
Jul 24 03:30:34.030196 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:34.030161    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-cz6z2"
Jul 24 03:30:51.631932 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.631854    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"]
Jul 24 03:30:51.634162 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.634140    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.636750 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.636723    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 03:30:51.636865 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.636756    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p7qhl\""
Jul 24 03:30:51.637296 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.637275    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 03:30:51.637418 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.637373    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 24 03:30:51.647773 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.647739    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"]
Jul 24 03:30:51.727732 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.727697    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.727912 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.727741    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.727912 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.727794    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.727912 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.727822    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.728017 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.727920    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpnwt\" (UniqueName: \"kubernetes.io/projected/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-kube-api-access-wpnwt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.728017 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.727962    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.829356 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.829321    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpnwt\" (UniqueName: \"kubernetes.io/projected/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-kube-api-access-wpnwt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.829356 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.829368    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.829635 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.829414    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.829635 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.829450    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.829635 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.829470    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.829635 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.829493    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.829952 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.829925    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.829952 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.829942    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.830077 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.829978    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.831979 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.831959    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.832128 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.832111    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.837162 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.837142    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpnwt\" (UniqueName: \"kubernetes.io/projected/da8fdad3-4aea-4fd4-8b38-04f1bc0215f1-kube-api-access-wpnwt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-9xc96\" (UID: \"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:51.944711 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:51.944663    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:30:52.100714 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:52.100689    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"]
Jul 24 03:30:52.103387 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:30:52.103353    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda8fdad3_4aea_4fd4_8b38_04f1bc0215f1.slice/crio-540edab81871eed4848465d158edfb17b5ceb2cd7dc1d4188de3455cb6b4149e WatchSource:0}: Error finding container 540edab81871eed4848465d158edfb17b5ceb2cd7dc1d4188de3455cb6b4149e: Status 404 returned error can't find the container with id 540edab81871eed4848465d158edfb17b5ceb2cd7dc1d4188de3455cb6b4149e
Jul 24 03:30:52.166785 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:52.166743    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96" event={"ID":"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1","Type":"ContainerStarted","Data":"540edab81871eed4848465d158edfb17b5ceb2cd7dc1d4188de3455cb6b4149e"}
Jul 24 03:30:53.173580 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:53.173540    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96" event={"ID":"da8fdad3-4aea-4fd4-8b38-04f1bc0215f1","Type":"ContainerStarted","Data":"be08310e7001e7ec0cd00120dc6b5dff6550db80a66fa682b7b8c2d0857999ec"}
Jul 24 03:30:53.194663 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:30:53.194607    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96" podStartSLOduration=1.26028615 podStartE2EDuration="2.194569643s" totalImagesPullingTime="934.283493ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:30:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:30:53.192388558 +0000 UTC m=+808.155481838" watchObservedRunningTime="2026-07-24 03:30:53.194569643 +0000 UTC m=+808.157662921"
Jul 24 03:31:01.945787 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:01.945740    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:31:01.946475 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:01.945822    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:31:01.958447 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:01.958393    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:31:02.225144 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:02.225056    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-9xc96"
Jul 24 03:31:06.439175 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.439095    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"]
Jul 24 03:31:06.444075 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.444054    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.446542 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.446504    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 24 03:31:06.453397 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.453371    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"]
Jul 24 03:31:06.574792 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.574749    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cgtv\" (UniqueName: \"kubernetes.io/projected/c69e11bf-47be-45ef-aec3-e030916a8c52-kube-api-access-4cgtv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.574792 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.574791    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c69e11bf-47be-45ef-aec3-e030916a8c52-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.575007 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.574889    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c69e11bf-47be-45ef-aec3-e030916a8c52-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.575007 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.574936    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c69e11bf-47be-45ef-aec3-e030916a8c52-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.575073 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.575017    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c69e11bf-47be-45ef-aec3-e030916a8c52-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.575073 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.575049    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c69e11bf-47be-45ef-aec3-e030916a8c52-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.676273 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.676236    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cgtv\" (UniqueName: \"kubernetes.io/projected/c69e11bf-47be-45ef-aec3-e030916a8c52-kube-api-access-4cgtv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.676273 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.676274    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c69e11bf-47be-45ef-aec3-e030916a8c52-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.676577 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.676354    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c69e11bf-47be-45ef-aec3-e030916a8c52-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.676577 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.676402    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c69e11bf-47be-45ef-aec3-e030916a8c52-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.676577 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.676486    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c69e11bf-47be-45ef-aec3-e030916a8c52-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.676577 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.676513    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c69e11bf-47be-45ef-aec3-e030916a8c52-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.676773 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.676705    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c69e11bf-47be-45ef-aec3-e030916a8c52-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.676773 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.676752    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c69e11bf-47be-45ef-aec3-e030916a8c52-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.676886 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.676869    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c69e11bf-47be-45ef-aec3-e030916a8c52-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.678898 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.678870    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c69e11bf-47be-45ef-aec3-e030916a8c52-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.679237 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.679214    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c69e11bf-47be-45ef-aec3-e030916a8c52-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.684031 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.684002    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cgtv\" (UniqueName: \"kubernetes.io/projected/c69e11bf-47be-45ef-aec3-e030916a8c52-kube-api-access-4cgtv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz\" (UID: \"c69e11bf-47be-45ef-aec3-e030916a8c52\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.755421 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.755339    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:06.924084 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:06.924060    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"]
Jul 24 03:31:06.926395 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:31:06.926366    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc69e11bf_47be_45ef_aec3_e030916a8c52.slice/crio-77a1acb81f687ca520d275d407fd90c8c468a4bfb57e2a030d187c1914806e23 WatchSource:0}: Error finding container 77a1acb81f687ca520d275d407fd90c8c468a4bfb57e2a030d187c1914806e23: Status 404 returned error can't find the container with id 77a1acb81f687ca520d275d407fd90c8c468a4bfb57e2a030d187c1914806e23
Jul 24 03:31:07.238176 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:07.238144    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz" event={"ID":"c69e11bf-47be-45ef-aec3-e030916a8c52","Type":"ContainerStarted","Data":"a0bae529dd033672d6832def8862a732382a6a776940b6e44cb211e52f2e1135"}
Jul 24 03:31:07.238355 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:07.238194    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz" event={"ID":"c69e11bf-47be-45ef-aec3-e030916a8c52","Type":"ContainerStarted","Data":"77a1acb81f687ca520d275d407fd90c8c468a4bfb57e2a030d187c1914806e23"}
Jul 24 03:31:07.258065 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:07.258006    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz" podStartSLOduration=1.1044859439999999 podStartE2EDuration="1.257986439s" totalImagesPullingTime="153.500495ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:31:07.255318631 +0000 UTC m=+822.218411930" watchObservedRunningTime="2026-07-24 03:31:07.257986439 +0000 UTC m=+822.221079726"
Jul 24 03:31:16.755928 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:16.755875    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:16.756411 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:16.755959    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:16.768729 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:16.768702    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:17.296077 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:17.296047    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz"
Jul 24 03:31:30.634953 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.634920    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"]
Jul 24 03:31:30.639947 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.639927    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.642306 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.642284    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 24 03:31:30.652831 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.652805    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"]
Jul 24 03:31:30.700236 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.700198    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4019c381-660b-4515-a6f8-1db607a0ac82-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.700236 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.700238    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4019c381-660b-4515-a6f8-1db607a0ac82-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.700470 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.700286    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4019c381-660b-4515-a6f8-1db607a0ac82-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.700470 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.700377    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4019c381-660b-4515-a6f8-1db607a0ac82-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.700470 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.700412    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g524j\" (UniqueName: \"kubernetes.io/projected/4019c381-660b-4515-a6f8-1db607a0ac82-kube-api-access-g524j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.700609 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.700504    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4019c381-660b-4515-a6f8-1db607a0ac82-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.801563 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.801531    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4019c381-660b-4515-a6f8-1db607a0ac82-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.801563 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.801567    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g524j\" (UniqueName: \"kubernetes.io/projected/4019c381-660b-4515-a6f8-1db607a0ac82-kube-api-access-g524j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.801798 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.801614    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4019c381-660b-4515-a6f8-1db607a0ac82-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.801798 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.801646    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4019c381-660b-4515-a6f8-1db607a0ac82-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.801798 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.801666    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4019c381-660b-4515-a6f8-1db607a0ac82-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.801798 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.801724    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4019c381-660b-4515-a6f8-1db607a0ac82-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.802127 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.802095    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4019c381-660b-4515-a6f8-1db607a0ac82-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.802127 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.802113    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4019c381-660b-4515-a6f8-1db607a0ac82-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.802286 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.802151    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4019c381-660b-4515-a6f8-1db607a0ac82-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.804106 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.804080    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4019c381-660b-4515-a6f8-1db607a0ac82-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.804267 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.804252    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4019c381-660b-4515-a6f8-1db607a0ac82-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.811025 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.811001    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g524j\" (UniqueName: \"kubernetes.io/projected/4019c381-660b-4515-a6f8-1db607a0ac82-kube-api-access-g524j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-zg2zz\" (UID: \"4019c381-660b-4515-a6f8-1db607a0ac82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:30.952384 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:30.952347    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:31.099378 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:31.099342    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"]
Jul 24 03:31:31.100820 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:31:31.100789    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4019c381_660b_4515_a6f8_1db607a0ac82.slice/crio-7f5969a77d569b8c39beb02797bc1c147b57b4e5b319eb32dd6099562c7544c7 WatchSource:0}: Error finding container 7f5969a77d569b8c39beb02797bc1c147b57b4e5b319eb32dd6099562c7544c7: Status 404 returned error can't find the container with id 7f5969a77d569b8c39beb02797bc1c147b57b4e5b319eb32dd6099562c7544c7
Jul 24 03:31:31.102679 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:31.102659    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 03:31:31.354555 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:31.354522    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz" event={"ID":"4019c381-660b-4515-a6f8-1db607a0ac82","Type":"ContainerStarted","Data":"ee1ff52be329491013517192847803ff2a581feca8341f25e16bd04c36be28b9"}
Jul 24 03:31:31.354735 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:31.354589    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz" event={"ID":"4019c381-660b-4515-a6f8-1db607a0ac82","Type":"ContainerStarted","Data":"7f5969a77d569b8c39beb02797bc1c147b57b4e5b319eb32dd6099562c7544c7"}
Jul 24 03:31:31.370790 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:31.370741    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz" podStartSLOduration=1.196795686 podStartE2EDuration="1.370722262s" totalImagesPullingTime="173.926576ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:31:31.369779673 +0000 UTC m=+846.332872962" watchObservedRunningTime="2026-07-24 03:31:31.370722262 +0000 UTC m=+846.333815548"
Jul 24 03:31:40.953105 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:40.953064    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:40.953625 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:40.953121    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:40.965944 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:40.965919    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:31:41.420381 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:31:41.420352    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-zg2zz"
Jul 24 03:32:25.482212 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:32:25.482180    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:32:25.483273 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:32:25.483248    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:37:25.511727 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:37:25.511696    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:37:25.514859 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:37:25.514833    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:39:43.680947 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:43.680910    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"]
Jul 24 03:39:43.683628 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:43.681103    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm" podUID="df962657-4324-4884-ad07-012c86cdb09b" containerName="manager" containerID="cri-o://38b231a7a8352c00cf4fa4ee4505a31c695b476f048be46e1125dd688980cee9" gracePeriod=10
Jul 24 03:39:45.760939 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:45.760908    2574 generic.go:356] "Generic (PLEG): container finished" podID="df962657-4324-4884-ad07-012c86cdb09b" containerID="38b231a7a8352c00cf4fa4ee4505a31c695b476f048be46e1125dd688980cee9" exitCode=0
Jul 24 03:39:45.761299 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:45.760990    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm" event={"ID":"df962657-4324-4884-ad07-012c86cdb09b","Type":"ContainerDied","Data":"38b231a7a8352c00cf4fa4ee4505a31c695b476f048be46e1125dd688980cee9"}
Jul 24 03:39:45.845805 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:45.845776    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"
Jul 24 03:39:45.937468 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:45.937417    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df962657-4324-4884-ad07-012c86cdb09b-extensions-socket-volume\") pod \"df962657-4324-4884-ad07-012c86cdb09b\" (UID: \"df962657-4324-4884-ad07-012c86cdb09b\") "
Jul 24 03:39:45.937634 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:45.937552    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dw95x\" (UniqueName: \"kubernetes.io/projected/df962657-4324-4884-ad07-012c86cdb09b-kube-api-access-dw95x\") pod \"df962657-4324-4884-ad07-012c86cdb09b\" (UID: \"df962657-4324-4884-ad07-012c86cdb09b\") "
Jul 24 03:39:45.937799 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:45.937774    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df962657-4324-4884-ad07-012c86cdb09b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "df962657-4324-4884-ad07-012c86cdb09b" (UID: "df962657-4324-4884-ad07-012c86cdb09b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 03:39:45.939871 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:45.939843    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df962657-4324-4884-ad07-012c86cdb09b-kube-api-access-dw95x" (OuterVolumeSpecName: "kube-api-access-dw95x") pod "df962657-4324-4884-ad07-012c86cdb09b" (UID: "df962657-4324-4884-ad07-012c86cdb09b"). InnerVolumeSpecName "kube-api-access-dw95x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:39:46.038220 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:46.038176    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw95x\" (UniqueName: \"kubernetes.io/projected/df962657-4324-4884-ad07-012c86cdb09b-kube-api-access-dw95x\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:39:46.038220 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:46.038215    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df962657-4324-4884-ad07-012c86cdb09b-extensions-socket-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:39:46.769536 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:46.769503    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"
Jul 24 03:39:46.769927 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:46.769544    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm" event={"ID":"df962657-4324-4884-ad07-012c86cdb09b","Type":"ContainerDied","Data":"ff4b31f18dc14d4e17fee96c38de27ce1a0b41416d311f507a44c6a8bceeaaad"}
Jul 24 03:39:46.769927 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:46.769613    2574 scope.go:117] "RemoveContainer" containerID="38b231a7a8352c00cf4fa4ee4505a31c695b476f048be46e1125dd688980cee9"
Jul 24 03:39:46.792258 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:46.792230    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"]
Jul 24 03:39:46.796734 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:46.796704    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-p6kfm"]
Jul 24 03:39:47.574079 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:39:47.574040    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df962657-4324-4884-ad07-012c86cdb09b" path="/var/lib/kubelet/pods/df962657-4324-4884-ad07-012c86cdb09b/volumes"
Jul 24 03:40:44.888091 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:44.888059    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t"]
Jul 24 03:40:44.888606 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:44.888440    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df962657-4324-4884-ad07-012c86cdb09b" containerName="manager"
Jul 24 03:40:44.888606 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:44.888456    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="df962657-4324-4884-ad07-012c86cdb09b" containerName="manager"
Jul 24 03:40:44.888606 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:44.888525    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="df962657-4324-4884-ad07-012c86cdb09b" containerName="manager"
Jul 24 03:40:44.892679 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:44.892663    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t"
Jul 24 03:40:44.894886 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:44.894862    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 03:40:44.895026 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:44.894889    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 03:40:44.895026 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:44.894874    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-44f58\""
Jul 24 03:40:44.903930 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:44.903909    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t"]
Jul 24 03:40:44.978867 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:44.978830    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2hw5\" (UniqueName: \"kubernetes.io/projected/ef1db0c8-bc87-413a-b402-115057ea1215-kube-api-access-z2hw5\") pod \"kuadrant-operator-controller-manager-86878c769f-6268t\" (UID: \"ef1db0c8-bc87-413a-b402-115057ea1215\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t"
Jul 24 03:40:44.979046 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:44.978903    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef1db0c8-bc87-413a-b402-115057ea1215-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6268t\" (UID: \"ef1db0c8-bc87-413a-b402-115057ea1215\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t"
Jul 24 03:40:45.079821 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:45.079772    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef1db0c8-bc87-413a-b402-115057ea1215-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6268t\" (UID: \"ef1db0c8-bc87-413a-b402-115057ea1215\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t"
Jul 24 03:40:45.080001 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:45.079926    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2hw5\" (UniqueName: \"kubernetes.io/projected/ef1db0c8-bc87-413a-b402-115057ea1215-kube-api-access-z2hw5\") pod \"kuadrant-operator-controller-manager-86878c769f-6268t\" (UID: \"ef1db0c8-bc87-413a-b402-115057ea1215\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t"
Jul 24 03:40:45.080149 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:45.080132    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef1db0c8-bc87-413a-b402-115057ea1215-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6268t\" (UID: \"ef1db0c8-bc87-413a-b402-115057ea1215\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t"
Jul 24 03:40:45.088814 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:45.088783    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2hw5\" (UniqueName: \"kubernetes.io/projected/ef1db0c8-bc87-413a-b402-115057ea1215-kube-api-access-z2hw5\") pod \"kuadrant-operator-controller-manager-86878c769f-6268t\" (UID: \"ef1db0c8-bc87-413a-b402-115057ea1215\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t"
Jul 24 03:40:45.203855 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:45.203754    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t"
Jul 24 03:40:45.353376 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:45.353348    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t"]
Jul 24 03:40:45.356116 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:40:45.356085    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef1db0c8_bc87_413a_b402_115057ea1215.slice/crio-b4abf437d16f44937b397881235ccf13647764a6367de579d25c83f15af42471 WatchSource:0}: Error finding container b4abf437d16f44937b397881235ccf13647764a6367de579d25c83f15af42471: Status 404 returned error can't find the container with id b4abf437d16f44937b397881235ccf13647764a6367de579d25c83f15af42471
Jul 24 03:40:46.047155 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:46.047121    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t" event={"ID":"ef1db0c8-bc87-413a-b402-115057ea1215","Type":"ContainerStarted","Data":"584025cea44e67490eac1205c94b449f92604b72ea91da0829d6a2eab15dd767"}
Jul 24 03:40:46.047592 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:46.047174    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t" event={"ID":"ef1db0c8-bc87-413a-b402-115057ea1215","Type":"ContainerStarted","Data":"b4abf437d16f44937b397881235ccf13647764a6367de579d25c83f15af42471"}
Jul 24 03:40:46.047592 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:46.047290    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t"
Jul 24 03:40:46.071398 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:46.071350    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t" podStartSLOduration=2.071332093 podStartE2EDuration="2.071332093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:40:46.068958054 +0000 UTC m=+1401.032051342" watchObservedRunningTime="2026-07-24 03:40:46.071332093 +0000 UTC m=+1401.034425379"
Jul 24 03:40:57.054353 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:40:57.054325    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6268t"
Jul 24 03:42:25.541167 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:42:25.541138    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:42:25.546681 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:42:25.546661    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:47:25.571761 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:25.571728    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:47:25.577369 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:25.577342    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:47:49.707811 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.707774    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"]
Jul 24 03:47:49.711007 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.710989    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"
Jul 24 03:47:49.719662 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.719589    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-3b0f86fb-dockercfg-tzwns\""
Jul 24 03:47:49.721281 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.721263    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"]
Jul 24 03:47:49.723391 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.723376    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-3b0f86fb\""
Jul 24 03:47:49.724362 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.724348    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"
Jul 24 03:47:49.732278 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.732250    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"]
Jul 24 03:47:49.744326 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.744297    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"]
Jul 24 03:47:49.835949 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.835906    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr\" (UID: \"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"
Jul 24 03:47:49.836156 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.836048    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8m2g\" (UniqueName: \"kubernetes.io/projected/20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d-kube-api-access-s8m2g\") pod \"payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr\" (UID: \"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"
Jul 24 03:47:49.836156 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.836107    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-278dj\" (UniqueName: \"kubernetes.io/projected/042d1f91-ba69-48e3-9808-f3d403259b66-kube-api-access-278dj\") pod \"payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884\" (UID: \"042d1f91-ba69-48e3-9808-f3d403259b66\") " pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"
Jul 24 03:47:49.836273 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.836170    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/042d1f91-ba69-48e3-9808-f3d403259b66-plugins-config-volume\") pod \"payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884\" (UID: \"042d1f91-ba69-48e3-9808-f3d403259b66\") " pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"
Jul 24 03:47:49.937343 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.937299    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-278dj\" (UniqueName: \"kubernetes.io/projected/042d1f91-ba69-48e3-9808-f3d403259b66-kube-api-access-278dj\") pod \"payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884\" (UID: \"042d1f91-ba69-48e3-9808-f3d403259b66\") " pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"
Jul 24 03:47:49.937343 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.937350    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/042d1f91-ba69-48e3-9808-f3d403259b66-plugins-config-volume\") pod \"payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884\" (UID: \"042d1f91-ba69-48e3-9808-f3d403259b66\") " pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"
Jul 24 03:47:49.937657 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.937402    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr\" (UID: \"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"
Jul 24 03:47:49.937657 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.937500    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8m2g\" (UniqueName: \"kubernetes.io/projected/20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d-kube-api-access-s8m2g\") pod \"payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr\" (UID: \"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"
Jul 24 03:47:49.938220 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.938199    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr\" (UID: \"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"
Jul 24 03:47:49.938357 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.938331    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/042d1f91-ba69-48e3-9808-f3d403259b66-plugins-config-volume\") pod \"payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884\" (UID: \"042d1f91-ba69-48e3-9808-f3d403259b66\") " pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"
Jul 24 03:47:49.970114 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.970033    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8m2g\" (UniqueName: \"kubernetes.io/projected/20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d-kube-api-access-s8m2g\") pod \"payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr\" (UID: \"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"
Jul 24 03:47:49.989605 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:49.989567    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-278dj\" (UniqueName: \"kubernetes.io/projected/042d1f91-ba69-48e3-9808-f3d403259b66-kube-api-access-278dj\") pod \"payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884\" (UID: \"042d1f91-ba69-48e3-9808-f3d403259b66\") " pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"
Jul 24 03:47:50.022216 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:50.022176    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"
Jul 24 03:47:50.034447 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:50.034391    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"
Jul 24 03:47:50.184977 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:50.184946    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"]
Jul 24 03:47:50.187492 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:47:50.187466    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d6c2e1_9f9f_4ce9_a23c_18c20294cc0d.slice/crio-caf3c4653ea34502ad4d7a026b0c519e9413bdfc678c16bbb9659e873a25f01e WatchSource:0}: Error finding container caf3c4653ea34502ad4d7a026b0c519e9413bdfc678c16bbb9659e873a25f01e: Status 404 returned error can't find the container with id caf3c4653ea34502ad4d7a026b0c519e9413bdfc678c16bbb9659e873a25f01e
Jul 24 03:47:50.205344 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:50.205318    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"]
Jul 24 03:47:50.207182 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:47:50.207153    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod042d1f91_ba69_48e3_9808_f3d403259b66.slice/crio-beb8e0ed4362efbc2c6944d379fdd6ef79786eac84aa112bf9761d4b9875572e WatchSource:0}: Error finding container beb8e0ed4362efbc2c6944d379fdd6ef79786eac84aa112bf9761d4b9875572e: Status 404 returned error can't find the container with id beb8e0ed4362efbc2c6944d379fdd6ef79786eac84aa112bf9761d4b9875572e
Jul 24 03:47:51.114952 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:51.114915    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr" event={"ID":"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d","Type":"ContainerStarted","Data":"9e5455f3ac077179654953dc93f091069c0fa33b7dde7c5b34cbe8073628ae6d"}
Jul 24 03:47:51.115399 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:51.115038    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr" event={"ID":"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d","Type":"ContainerStarted","Data":"caf3c4653ea34502ad4d7a026b0c519e9413bdfc678c16bbb9659e873a25f01e"}
Jul 24 03:47:51.115399 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:51.115069    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"
Jul 24 03:47:51.116378 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:51.116356    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884" event={"ID":"042d1f91-ba69-48e3-9808-f3d403259b66","Type":"ContainerStarted","Data":"3ab95ca1d77fdd1755d36292cc318b80f7855e916010a10fdfc4f4dfa3c611ef"}
Jul 24 03:47:51.116464 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:51.116403    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"
Jul 24 03:47:51.116464 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:51.116418    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884" event={"ID":"042d1f91-ba69-48e3-9808-f3d403259b66","Type":"ContainerStarted","Data":"beb8e0ed4362efbc2c6944d379fdd6ef79786eac84aa112bf9761d4b9875572e"}
Jul 24 03:47:51.133677 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:51.133611    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr" podStartSLOduration=2.133590955 podStartE2EDuration="2.133590955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:47:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:47:51.13186651 +0000 UTC m=+1826.094959797" watchObservedRunningTime="2026-07-24 03:47:51.133590955 +0000 UTC m=+1826.096684243"
Jul 24 03:47:51.149912 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:47:51.149832    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884" podStartSLOduration=2.149811313 podStartE2EDuration="2.149811313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:47:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:47:51.148860676 +0000 UTC m=+1826.111953963" watchObservedRunningTime="2026-07-24 03:47:51.149811313 +0000 UTC m=+1826.112904600"
Jul 24 03:48:02.122817 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:02.122779    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"
Jul 24 03:48:02.123273 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:02.122901    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"
Jul 24 03:48:14.541167 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:14.541128    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552"]
Jul 24 03:48:14.544694 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:14.544670    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552"
Jul 24 03:48:14.546992 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:14.546965    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 03:48:14.547100 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:14.547001    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 03:48:14.552277 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:14.552243    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552"]
Jul 24 03:48:14.676142 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:14.676106    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8efba3f-af61-408d-a8ca-1b62d676d5c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552\" (UID: \"f8efba3f-af61-408d-a8ca-1b62d676d5c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552"
Jul 24 03:48:14.777580 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:14.777543    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8efba3f-af61-408d-a8ca-1b62d676d5c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552\" (UID: \"f8efba3f-af61-408d-a8ca-1b62d676d5c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552"
Jul 24 03:48:14.778154 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:14.778134    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8efba3f-af61-408d-a8ca-1b62d676d5c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552\" (UID: \"f8efba3f-af61-408d-a8ca-1b62d676d5c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552"
Jul 24 03:48:14.857268 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:14.857187    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552"
Jul 24 03:48:14.995697 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:14.995661    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552"]
Jul 24 03:48:14.998571 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:48:14.998544    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8efba3f_af61_408d_a8ca_1b62d676d5c7.slice/crio-a1374a2fc01bfc0b3d6c3553f87b2cdb83ef7b12e33aeb2525fd9732b4878df4 WatchSource:0}: Error finding container a1374a2fc01bfc0b3d6c3553f87b2cdb83ef7b12e33aeb2525fd9732b4878df4: Status 404 returned error can't find the container with id a1374a2fc01bfc0b3d6c3553f87b2cdb83ef7b12e33aeb2525fd9732b4878df4
Jul 24 03:48:15.000467 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:15.000452    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 03:48:15.235739 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:15.235694    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552" event={"ID":"f8efba3f-af61-408d-a8ca-1b62d676d5c7","Type":"ContainerStarted","Data":"a1374a2fc01bfc0b3d6c3553f87b2cdb83ef7b12e33aeb2525fd9732b4878df4"}
Jul 24 03:48:18.257373 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:18.257337    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552" event={"ID":"f8efba3f-af61-408d-a8ca-1b62d676d5c7","Type":"ContainerStarted","Data":"89841280c3aaf3e32154576a377b193252398ba228c6e50b9a903f1b43d1b0ff"}
Jul 24 03:48:18.273199 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:18.273140    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552" podStartSLOduration=1.552319386 podStartE2EDuration="4.27312213s" totalImagesPullingTime="2.720802744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:48:18.271009663 +0000 UTC m=+1853.234102951" watchObservedRunningTime="2026-07-24 03:48:18.27312213 +0000 UTC m=+1853.236215417"
Jul 24 03:48:19.263970 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:19.263934    2574 generic.go:356] "Generic (PLEG): container finished" podID="f8efba3f-af61-408d-a8ca-1b62d676d5c7" containerID="89841280c3aaf3e32154576a377b193252398ba228c6e50b9a903f1b43d1b0ff" exitCode=0
Jul 24 03:48:19.264481 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:19.264046    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552" event={"ID":"f8efba3f-af61-408d-a8ca-1b62d676d5c7","Type":"ContainerDied","Data":"89841280c3aaf3e32154576a377b193252398ba228c6e50b9a903f1b43d1b0ff"}
Jul 24 03:48:20.409052 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:20.409027    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552"
Jul 24 03:48:20.429651 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:20.429620    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8efba3f-af61-408d-a8ca-1b62d676d5c7-maas-api-service-ca\") pod \"f8efba3f-af61-408d-a8ca-1b62d676d5c7\" (UID: \"f8efba3f-af61-408d-a8ca-1b62d676d5c7\") "
Jul 24 03:48:20.430049 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:20.430023    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8efba3f-af61-408d-a8ca-1b62d676d5c7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f8efba3f-af61-408d-a8ca-1b62d676d5c7" (UID: "f8efba3f-af61-408d-a8ca-1b62d676d5c7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:48:20.531139 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:20.531057    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8efba3f-af61-408d-a8ca-1b62d676d5c7-maas-api-service-ca\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:48:21.276262 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:21.276234    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552"
Jul 24 03:48:21.276262 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:21.276235    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552" event={"ID":"f8efba3f-af61-408d-a8ca-1b62d676d5c7","Type":"ContainerDied","Data":"a1374a2fc01bfc0b3d6c3553f87b2cdb83ef7b12e33aeb2525fd9732b4878df4"}
Jul 24 03:48:21.276513 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:21.276312    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1374a2fc01bfc0b3d6c3553f87b2cdb83ef7b12e33aeb2525fd9732b4878df4"
Jul 24 03:48:24.982113 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:24.982078    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"]
Jul 24 03:48:24.982717 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:24.982302    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884" podUID="042d1f91-ba69-48e3-9808-f3d403259b66" containerName="payload-processing" containerID="cri-o://3ab95ca1d77fdd1755d36292cc318b80f7855e916010a10fdfc4f4dfa3c611ef" gracePeriod=30
Jul 24 03:48:24.989117 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:24.989083    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"]
Jul 24 03:48:24.989378 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:24.989320    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr" podUID="20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d" containerName="payload-pre-processing" containerID="cri-o://9e5455f3ac077179654953dc93f091069c0fa33b7dde7c5b34cbe8073628ae6d" gracePeriod=30
Jul 24 03:48:25.275557 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.275527    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"
Jul 24 03:48:25.278285 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.278265    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"
Jul 24 03:48:25.298223 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.298188    2574 generic.go:356] "Generic (PLEG): container finished" podID="20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d" containerID="9e5455f3ac077179654953dc93f091069c0fa33b7dde7c5b34cbe8073628ae6d" exitCode=0
Jul 24 03:48:25.298439 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.298239    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"
Jul 24 03:48:25.298439 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.298274    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr" event={"ID":"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d","Type":"ContainerDied","Data":"9e5455f3ac077179654953dc93f091069c0fa33b7dde7c5b34cbe8073628ae6d"}
Jul 24 03:48:25.298439 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.298332    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr" event={"ID":"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d","Type":"ContainerDied","Data":"caf3c4653ea34502ad4d7a026b0c519e9413bdfc678c16bbb9659e873a25f01e"}
Jul 24 03:48:25.298439 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.298365    2574 scope.go:117] "RemoveContainer" containerID="9e5455f3ac077179654953dc93f091069c0fa33b7dde7c5b34cbe8073628ae6d"
Jul 24 03:48:25.299924 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.299790    2574 generic.go:356] "Generic (PLEG): container finished" podID="042d1f91-ba69-48e3-9808-f3d403259b66" containerID="3ab95ca1d77fdd1755d36292cc318b80f7855e916010a10fdfc4f4dfa3c611ef" exitCode=0
Jul 24 03:48:25.299924 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.299862    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884" event={"ID":"042d1f91-ba69-48e3-9808-f3d403259b66","Type":"ContainerDied","Data":"3ab95ca1d77fdd1755d36292cc318b80f7855e916010a10fdfc4f4dfa3c611ef"}
Jul 24 03:48:25.299924 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.299898    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884" event={"ID":"042d1f91-ba69-48e3-9808-f3d403259b66","Type":"ContainerDied","Data":"beb8e0ed4362efbc2c6944d379fdd6ef79786eac84aa112bf9761d4b9875572e"}
Jul 24 03:48:25.300112 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.300009    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"
Jul 24 03:48:25.308649 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.308541    2574 scope.go:117] "RemoveContainer" containerID="9e5455f3ac077179654953dc93f091069c0fa33b7dde7c5b34cbe8073628ae6d"
Jul 24 03:48:25.309094 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:48:25.309067    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e5455f3ac077179654953dc93f091069c0fa33b7dde7c5b34cbe8073628ae6d\": container with ID starting with 9e5455f3ac077179654953dc93f091069c0fa33b7dde7c5b34cbe8073628ae6d not found: ID does not exist" containerID="9e5455f3ac077179654953dc93f091069c0fa33b7dde7c5b34cbe8073628ae6d"
Jul 24 03:48:25.309188 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.309107    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e5455f3ac077179654953dc93f091069c0fa33b7dde7c5b34cbe8073628ae6d"} err="failed to get container status \"9e5455f3ac077179654953dc93f091069c0fa33b7dde7c5b34cbe8073628ae6d\": rpc error: code = NotFound desc = could not find container \"9e5455f3ac077179654953dc93f091069c0fa33b7dde7c5b34cbe8073628ae6d\": container with ID starting with 9e5455f3ac077179654953dc93f091069c0fa33b7dde7c5b34cbe8073628ae6d not found: ID does not exist"
Jul 24 03:48:25.309188 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.309132    2574 scope.go:117] "RemoveContainer" containerID="3ab95ca1d77fdd1755d36292cc318b80f7855e916010a10fdfc4f4dfa3c611ef"
Jul 24 03:48:25.318120 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.318101    2574 scope.go:117] "RemoveContainer" containerID="3ab95ca1d77fdd1755d36292cc318b80f7855e916010a10fdfc4f4dfa3c611ef"
Jul 24 03:48:25.318445 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:48:25.318409    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ab95ca1d77fdd1755d36292cc318b80f7855e916010a10fdfc4f4dfa3c611ef\": container with ID starting with 3ab95ca1d77fdd1755d36292cc318b80f7855e916010a10fdfc4f4dfa3c611ef not found: ID does not exist" containerID="3ab95ca1d77fdd1755d36292cc318b80f7855e916010a10fdfc4f4dfa3c611ef"
Jul 24 03:48:25.318515 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.318455    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ab95ca1d77fdd1755d36292cc318b80f7855e916010a10fdfc4f4dfa3c611ef"} err="failed to get container status \"3ab95ca1d77fdd1755d36292cc318b80f7855e916010a10fdfc4f4dfa3c611ef\": rpc error: code = NotFound desc = could not find container \"3ab95ca1d77fdd1755d36292cc318b80f7855e916010a10fdfc4f4dfa3c611ef\": container with ID starting with 3ab95ca1d77fdd1755d36292cc318b80f7855e916010a10fdfc4f4dfa3c611ef not found: ID does not exist"
Jul 24 03:48:25.373783 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.373747    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-278dj\" (UniqueName: \"kubernetes.io/projected/042d1f91-ba69-48e3-9808-f3d403259b66-kube-api-access-278dj\") pod \"042d1f91-ba69-48e3-9808-f3d403259b66\" (UID: \"042d1f91-ba69-48e3-9808-f3d403259b66\") "
Jul 24 03:48:25.373783 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.373788    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8m2g\" (UniqueName: \"kubernetes.io/projected/20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d-kube-api-access-s8m2g\") pod \"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d\" (UID: \"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d\") "
Jul 24 03:48:25.374046 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.373827    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/042d1f91-ba69-48e3-9808-f3d403259b66-plugins-config-volume\") pod \"042d1f91-ba69-48e3-9808-f3d403259b66\" (UID: \"042d1f91-ba69-48e3-9808-f3d403259b66\") "
Jul 24 03:48:25.374046 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.373852    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d-plugins-config-volume\") pod \"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d\" (UID: \"20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d\") "
Jul 24 03:48:25.374247 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.374224    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/042d1f91-ba69-48e3-9808-f3d403259b66-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "042d1f91-ba69-48e3-9808-f3d403259b66" (UID: "042d1f91-ba69-48e3-9808-f3d403259b66"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:48:25.374318 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.374301    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d" (UID: "20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:48:25.376126 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.376104    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d-kube-api-access-s8m2g" (OuterVolumeSpecName: "kube-api-access-s8m2g") pod "20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d" (UID: "20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d"). InnerVolumeSpecName "kube-api-access-s8m2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:48:25.376208 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.376165    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/042d1f91-ba69-48e3-9808-f3d403259b66-kube-api-access-278dj" (OuterVolumeSpecName: "kube-api-access-278dj") pod "042d1f91-ba69-48e3-9808-f3d403259b66" (UID: "042d1f91-ba69-48e3-9808-f3d403259b66"). InnerVolumeSpecName "kube-api-access-278dj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:48:25.475345 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.475313    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-278dj\" (UniqueName: \"kubernetes.io/projected/042d1f91-ba69-48e3-9808-f3d403259b66-kube-api-access-278dj\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:48:25.475345 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.475343    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8m2g\" (UniqueName: \"kubernetes.io/projected/20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d-kube-api-access-s8m2g\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:48:25.475345 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.475354    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/042d1f91-ba69-48e3-9808-f3d403259b66-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:48:25.475611 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.475363    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:48:25.625836 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.625807    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"]
Jul 24 03:48:25.627607 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.627587    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3b0f86fb-57dc899d6d-t62rr"]
Jul 24 03:48:25.637371 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.637342    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"]
Jul 24 03:48:25.639523 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:25.639500    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3b0f86fb-79788df8c8-vs884"]
Jul 24 03:48:27.572971 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:27.572937    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="042d1f91-ba69-48e3-9808-f3d403259b66" path="/var/lib/kubelet/pods/042d1f91-ba69-48e3-9808-f3d403259b66/volumes"
Jul 24 03:48:27.573482 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:27.573407    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d" path="/var/lib/kubelet/pods/20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d/volumes"
Jul 24 03:48:29.614273 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:29.614232    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552"]
Jul 24 03:48:29.616454 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:29.616401    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3b0f86fb-t6552"]
Jul 24 03:48:31.572810 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:31.572773    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8efba3f-af61-408d-a8ca-1b62d676d5c7" path="/var/lib/kubelet/pods/f8efba3f-af61-408d-a8ca-1b62d676d5c7/volumes"
Jul 24 03:48:34.037385 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.037341    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"]
Jul 24 03:48:34.037954 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.037937    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d" containerName="payload-pre-processing"
Jul 24 03:48:34.038006 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.037959    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d" containerName="payload-pre-processing"
Jul 24 03:48:34.038006 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.038002    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8efba3f-af61-408d-a8ca-1b62d676d5c7" containerName="revoke-keys"
Jul 24 03:48:34.038076 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.038010    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8efba3f-af61-408d-a8ca-1b62d676d5c7" containerName="revoke-keys"
Jul 24 03:48:34.038076 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.038022    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="042d1f91-ba69-48e3-9808-f3d403259b66" containerName="payload-processing"
Jul 24 03:48:34.038076 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.038031    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="042d1f91-ba69-48e3-9808-f3d403259b66" containerName="payload-processing"
Jul 24 03:48:34.038222 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.038125    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="20d6c2e1-9f9f-4ce9-a23c-18c20294cc0d" containerName="payload-pre-processing"
Jul 24 03:48:34.038222 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.038168    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="042d1f91-ba69-48e3-9808-f3d403259b66" containerName="payload-processing"
Jul 24 03:48:34.038222 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.038179    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8efba3f-af61-408d-a8ca-1b62d676d5c7" containerName="revoke-keys"
Jul 24 03:48:34.042968 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.042944    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"]
Jul 24 03:48:34.043133 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.043078    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"
Jul 24 03:48:34.045566 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.045545    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-781d8b17-dockercfg-vzw69\""
Jul 24 03:48:34.046015 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.045999    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-781d8b17\""
Jul 24 03:48:34.046639 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.046622    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"
Jul 24 03:48:34.051499 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.051467    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"]
Jul 24 03:48:34.060510 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.060479    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"]
Jul 24 03:48:34.156099 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.156061    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b70af663-07c7-4d7f-a36d-faeb9b2eeeab-plugins-config-volume\") pod \"payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q\" (UID: \"b70af663-07c7-4d7f-a36d-faeb9b2eeeab\") " pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"
Jul 24 03:48:34.156099 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.156106    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa293c3-c15d-4e2e-9c9b-a2020fb25032-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq\" (UID: \"2fa293c3-c15d-4e2e-9c9b-a2020fb25032\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"
Jul 24 03:48:34.156318 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.156195    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h9l2\" (UniqueName: \"kubernetes.io/projected/b70af663-07c7-4d7f-a36d-faeb9b2eeeab-kube-api-access-9h9l2\") pod \"payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q\" (UID: \"b70af663-07c7-4d7f-a36d-faeb9b2eeeab\") " pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"
Jul 24 03:48:34.156318 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.156217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqd2w\" (UniqueName: \"kubernetes.io/projected/2fa293c3-c15d-4e2e-9c9b-a2020fb25032-kube-api-access-gqd2w\") pod \"payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq\" (UID: \"2fa293c3-c15d-4e2e-9c9b-a2020fb25032\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"
Jul 24 03:48:34.257459 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.257392    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h9l2\" (UniqueName: \"kubernetes.io/projected/b70af663-07c7-4d7f-a36d-faeb9b2eeeab-kube-api-access-9h9l2\") pod \"payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q\" (UID: \"b70af663-07c7-4d7f-a36d-faeb9b2eeeab\") " pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"
Jul 24 03:48:34.257669 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.257467    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqd2w\" (UniqueName: \"kubernetes.io/projected/2fa293c3-c15d-4e2e-9c9b-a2020fb25032-kube-api-access-gqd2w\") pod \"payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq\" (UID: \"2fa293c3-c15d-4e2e-9c9b-a2020fb25032\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"
Jul 24 03:48:34.257669 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.257530    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b70af663-07c7-4d7f-a36d-faeb9b2eeeab-plugins-config-volume\") pod \"payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q\" (UID: \"b70af663-07c7-4d7f-a36d-faeb9b2eeeab\") " pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"
Jul 24 03:48:34.257669 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.257562    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa293c3-c15d-4e2e-9c9b-a2020fb25032-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq\" (UID: \"2fa293c3-c15d-4e2e-9c9b-a2020fb25032\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"
Jul 24 03:48:34.258238 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.258212    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b70af663-07c7-4d7f-a36d-faeb9b2eeeab-plugins-config-volume\") pod \"payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q\" (UID: \"b70af663-07c7-4d7f-a36d-faeb9b2eeeab\") " pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"
Jul 24 03:48:34.258342 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.258299    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa293c3-c15d-4e2e-9c9b-a2020fb25032-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq\" (UID: \"2fa293c3-c15d-4e2e-9c9b-a2020fb25032\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"
Jul 24 03:48:34.266510 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.266484    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqd2w\" (UniqueName: \"kubernetes.io/projected/2fa293c3-c15d-4e2e-9c9b-a2020fb25032-kube-api-access-gqd2w\") pod \"payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq\" (UID: \"2fa293c3-c15d-4e2e-9c9b-a2020fb25032\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"
Jul 24 03:48:34.266637 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.266510    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h9l2\" (UniqueName: \"kubernetes.io/projected/b70af663-07c7-4d7f-a36d-faeb9b2eeeab-kube-api-access-9h9l2\") pod \"payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q\" (UID: \"b70af663-07c7-4d7f-a36d-faeb9b2eeeab\") " pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"
Jul 24 03:48:34.357507 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.357382    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"
Jul 24 03:48:34.363705 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.363671    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"
Jul 24 03:48:34.523626 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.523599    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"]
Jul 24 03:48:34.523930 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:48:34.523904    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fa293c3_c15d_4e2e_9c9b_a2020fb25032.slice/crio-9f740e5eafa957a48e7f2c5c11f7205134b3f870da84e6359f1e804aa9c49688 WatchSource:0}: Error finding container 9f740e5eafa957a48e7f2c5c11f7205134b3f870da84e6359f1e804aa9c49688: Status 404 returned error can't find the container with id 9f740e5eafa957a48e7f2c5c11f7205134b3f870da84e6359f1e804aa9c49688
Jul 24 03:48:34.574584 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:48:34.574539    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb70af663_07c7_4d7f_a36d_faeb9b2eeeab.slice/crio-584537eddc3470d4e6c9dce679d69403c920c08a3527a3742ff6d3f3873eb65d WatchSource:0}: Error finding container 584537eddc3470d4e6c9dce679d69403c920c08a3527a3742ff6d3f3873eb65d: Status 404 returned error can't find the container with id 584537eddc3470d4e6c9dce679d69403c920c08a3527a3742ff6d3f3873eb65d
Jul 24 03:48:34.581393 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:34.581369    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"]
Jul 24 03:48:35.351418 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:35.351380    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q" event={"ID":"b70af663-07c7-4d7f-a36d-faeb9b2eeeab","Type":"ContainerStarted","Data":"19dc672da27d07ba066b7b210fa41ebcacd313c2b874171d852c26d44c23110b"}
Jul 24 03:48:35.351920 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:35.351604    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"
Jul 24 03:48:35.351920 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:35.351651    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q" event={"ID":"b70af663-07c7-4d7f-a36d-faeb9b2eeeab","Type":"ContainerStarted","Data":"584537eddc3470d4e6c9dce679d69403c920c08a3527a3742ff6d3f3873eb65d"}
Jul 24 03:48:35.353114 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:35.353090    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq" event={"ID":"2fa293c3-c15d-4e2e-9c9b-a2020fb25032","Type":"ContainerStarted","Data":"1c8075f82d4590fa8594479955bed836da028b79f8bb98f0c129645b25c75e7d"}
Jul 24 03:48:35.353228 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:35.353130    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq" event={"ID":"2fa293c3-c15d-4e2e-9c9b-a2020fb25032","Type":"ContainerStarted","Data":"9f740e5eafa957a48e7f2c5c11f7205134b3f870da84e6359f1e804aa9c49688"}
Jul 24 03:48:35.353228 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:35.353164    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"
Jul 24 03:48:35.365866 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:35.365801    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q" podStartSLOduration=1.365780603 podStartE2EDuration="1.365780603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:48:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:48:35.365160987 +0000 UTC m=+1870.328254286" watchObservedRunningTime="2026-07-24 03:48:35.365780603 +0000 UTC m=+1870.328873895"
Jul 24 03:48:35.380344 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:35.380287    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq" podStartSLOduration=1.380266709 podStartE2EDuration="1.380266709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:48:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:48:35.378794809 +0000 UTC m=+1870.341888095" watchObservedRunningTime="2026-07-24 03:48:35.380266709 +0000 UTC m=+1870.343359999"
Jul 24 03:48:46.359959 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:46.359881    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"
Jul 24 03:48:46.359959 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:46.359950    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"
Jul 24 03:48:58.388349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:58.388310    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg"]
Jul 24 03:48:58.391862 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:58.391834    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg"
Jul 24 03:48:58.394504 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:58.394483    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 03:48:58.394788 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:58.394775    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 03:48:58.397918 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:58.397806    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59f38a30-ac1d-47e8-b956-8c0f6488db81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg\" (UID: \"59f38a30-ac1d-47e8-b956-8c0f6488db81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg"
Jul 24 03:48:58.409801 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:58.401028    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg"]
Jul 24 03:48:58.498481 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:58.498413    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59f38a30-ac1d-47e8-b956-8c0f6488db81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg\" (UID: \"59f38a30-ac1d-47e8-b956-8c0f6488db81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg"
Jul 24 03:48:58.499097 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:58.499075    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59f38a30-ac1d-47e8-b956-8c0f6488db81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg\" (UID: \"59f38a30-ac1d-47e8-b956-8c0f6488db81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg"
Jul 24 03:48:58.705372 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:58.705268    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg"
Jul 24 03:48:58.870678 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:58.870651    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg"]
Jul 24 03:48:58.871628 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:48:58.871598    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59f38a30_ac1d_47e8_b956_8c0f6488db81.slice/crio-5102d0676494cebf504d80dc15eb8f5c117d66f769b6ad8530e39a4464bf7ebc WatchSource:0}: Error finding container 5102d0676494cebf504d80dc15eb8f5c117d66f769b6ad8530e39a4464bf7ebc: Status 404 returned error can't find the container with id 5102d0676494cebf504d80dc15eb8f5c117d66f769b6ad8530e39a4464bf7ebc
Jul 24 03:48:59.478099 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:59.478060    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg" event={"ID":"59f38a30-ac1d-47e8-b956-8c0f6488db81","Type":"ContainerStarted","Data":"d0d67010dd8f3c30572e2ac4033866c78c75e005e0fcb6b015302c2f146a56af"}
Jul 24 03:48:59.478620 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:59.478128    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg" event={"ID":"59f38a30-ac1d-47e8-b956-8c0f6488db81","Type":"ContainerStarted","Data":"5102d0676494cebf504d80dc15eb8f5c117d66f769b6ad8530e39a4464bf7ebc"}
Jul 24 03:48:59.494308 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:48:59.494250    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg" podStartSLOduration=1.494234279 podStartE2EDuration="1.494234279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:48:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:48:59.492698464 +0000 UTC m=+1894.455791751" watchObservedRunningTime="2026-07-24 03:48:59.494234279 +0000 UTC m=+1894.457327566"
Jul 24 03:49:00.484814 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:00.484780    2574 generic.go:356] "Generic (PLEG): container finished" podID="59f38a30-ac1d-47e8-b956-8c0f6488db81" containerID="d0d67010dd8f3c30572e2ac4033866c78c75e005e0fcb6b015302c2f146a56af" exitCode=0
Jul 24 03:49:00.485249 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:00.484841    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg" event={"ID":"59f38a30-ac1d-47e8-b956-8c0f6488db81","Type":"ContainerDied","Data":"d0d67010dd8f3c30572e2ac4033866c78c75e005e0fcb6b015302c2f146a56af"}
Jul 24 03:49:01.631770 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:01.631742    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg"
Jul 24 03:49:01.728667 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:01.728635    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59f38a30-ac1d-47e8-b956-8c0f6488db81-maas-api-service-ca\") pod \"59f38a30-ac1d-47e8-b956-8c0f6488db81\" (UID: \"59f38a30-ac1d-47e8-b956-8c0f6488db81\") "
Jul 24 03:49:01.729012 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:01.728988    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59f38a30-ac1d-47e8-b956-8c0f6488db81-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "59f38a30-ac1d-47e8-b956-8c0f6488db81" (UID: "59f38a30-ac1d-47e8-b956-8c0f6488db81"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:49:01.830317 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:01.830224    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59f38a30-ac1d-47e8-b956-8c0f6488db81-maas-api-service-ca\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:49:02.495602 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:02.495570    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg"
Jul 24 03:49:02.495602 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:02.495599    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg" event={"ID":"59f38a30-ac1d-47e8-b956-8c0f6488db81","Type":"ContainerDied","Data":"5102d0676494cebf504d80dc15eb8f5c117d66f769b6ad8530e39a4464bf7ebc"}
Jul 24 03:49:02.495824 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:02.495641    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5102d0676494cebf504d80dc15eb8f5c117d66f769b6ad8530e39a4464bf7ebc"
Jul 24 03:49:08.968054 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:08.968018    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"]
Jul 24 03:49:08.968803 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:08.968765    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q" podUID="b70af663-07c7-4d7f-a36d-faeb9b2eeeab" containerName="payload-processing" containerID="cri-o://19dc672da27d07ba066b7b210fa41ebcacd313c2b874171d852c26d44c23110b" gracePeriod=30
Jul 24 03:49:08.972702 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:08.972667    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"]
Jul 24 03:49:08.972930 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:08.972895    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq" podUID="2fa293c3-c15d-4e2e-9c9b-a2020fb25032" containerName="payload-pre-processing" containerID="cri-o://1c8075f82d4590fa8594479955bed836da028b79f8bb98f0c129645b25c75e7d" gracePeriod=30
Jul 24 03:49:09.279531 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.279501    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"
Jul 24 03:49:09.282340 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.282316    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"
Jul 24 03:49:09.406809 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.406769    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b70af663-07c7-4d7f-a36d-faeb9b2eeeab-plugins-config-volume\") pod \"b70af663-07c7-4d7f-a36d-faeb9b2eeeab\" (UID: \"b70af663-07c7-4d7f-a36d-faeb9b2eeeab\") "
Jul 24 03:49:09.407017 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.406820    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa293c3-c15d-4e2e-9c9b-a2020fb25032-plugins-config-volume\") pod \"2fa293c3-c15d-4e2e-9c9b-a2020fb25032\" (UID: \"2fa293c3-c15d-4e2e-9c9b-a2020fb25032\") "
Jul 24 03:49:09.407017 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.406849    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqd2w\" (UniqueName: \"kubernetes.io/projected/2fa293c3-c15d-4e2e-9c9b-a2020fb25032-kube-api-access-gqd2w\") pod \"2fa293c3-c15d-4e2e-9c9b-a2020fb25032\" (UID: \"2fa293c3-c15d-4e2e-9c9b-a2020fb25032\") "
Jul 24 03:49:09.407017 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.406949    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h9l2\" (UniqueName: \"kubernetes.io/projected/b70af663-07c7-4d7f-a36d-faeb9b2eeeab-kube-api-access-9h9l2\") pod \"b70af663-07c7-4d7f-a36d-faeb9b2eeeab\" (UID: \"b70af663-07c7-4d7f-a36d-faeb9b2eeeab\") "
Jul 24 03:49:09.407239 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.407204    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fa293c3-c15d-4e2e-9c9b-a2020fb25032-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2fa293c3-c15d-4e2e-9c9b-a2020fb25032" (UID: "2fa293c3-c15d-4e2e-9c9b-a2020fb25032"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:49:09.407382 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.407235    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b70af663-07c7-4d7f-a36d-faeb9b2eeeab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b70af663-07c7-4d7f-a36d-faeb9b2eeeab" (UID: "b70af663-07c7-4d7f-a36d-faeb9b2eeeab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:49:09.409204 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.409179    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b70af663-07c7-4d7f-a36d-faeb9b2eeeab-kube-api-access-9h9l2" (OuterVolumeSpecName: "kube-api-access-9h9l2") pod "b70af663-07c7-4d7f-a36d-faeb9b2eeeab" (UID: "b70af663-07c7-4d7f-a36d-faeb9b2eeeab"). InnerVolumeSpecName "kube-api-access-9h9l2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:49:09.409306 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.409282    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa293c3-c15d-4e2e-9c9b-a2020fb25032-kube-api-access-gqd2w" (OuterVolumeSpecName: "kube-api-access-gqd2w") pod "2fa293c3-c15d-4e2e-9c9b-a2020fb25032" (UID: "2fa293c3-c15d-4e2e-9c9b-a2020fb25032"). InnerVolumeSpecName "kube-api-access-gqd2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:49:09.508043 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.507952    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h9l2\" (UniqueName: \"kubernetes.io/projected/b70af663-07c7-4d7f-a36d-faeb9b2eeeab-kube-api-access-9h9l2\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:49:09.508043 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.507985    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b70af663-07c7-4d7f-a36d-faeb9b2eeeab-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:49:09.508043 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.507997    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa293c3-c15d-4e2e-9c9b-a2020fb25032-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:49:09.508043 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.508005    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqd2w\" (UniqueName: \"kubernetes.io/projected/2fa293c3-c15d-4e2e-9c9b-a2020fb25032-kube-api-access-gqd2w\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:49:09.530335 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.530303    2574 generic.go:356] "Generic (PLEG): container finished" podID="b70af663-07c7-4d7f-a36d-faeb9b2eeeab" containerID="19dc672da27d07ba066b7b210fa41ebcacd313c2b874171d852c26d44c23110b" exitCode=0
Jul 24 03:49:09.530520 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.530363    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"
Jul 24 03:49:09.530520 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.530379    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q" event={"ID":"b70af663-07c7-4d7f-a36d-faeb9b2eeeab","Type":"ContainerDied","Data":"19dc672da27d07ba066b7b210fa41ebcacd313c2b874171d852c26d44c23110b"}
Jul 24 03:49:09.530520 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.530493    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q" event={"ID":"b70af663-07c7-4d7f-a36d-faeb9b2eeeab","Type":"ContainerDied","Data":"584537eddc3470d4e6c9dce679d69403c920c08a3527a3742ff6d3f3873eb65d"}
Jul 24 03:49:09.530643 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.530525    2574 scope.go:117] "RemoveContainer" containerID="19dc672da27d07ba066b7b210fa41ebcacd313c2b874171d852c26d44c23110b"
Jul 24 03:49:09.531815 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.531795    2574 generic.go:356] "Generic (PLEG): container finished" podID="2fa293c3-c15d-4e2e-9c9b-a2020fb25032" containerID="1c8075f82d4590fa8594479955bed836da028b79f8bb98f0c129645b25c75e7d" exitCode=0
Jul 24 03:49:09.531923 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.531858    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"
Jul 24 03:49:09.531923 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.531880    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq" event={"ID":"2fa293c3-c15d-4e2e-9c9b-a2020fb25032","Type":"ContainerDied","Data":"1c8075f82d4590fa8594479955bed836da028b79f8bb98f0c129645b25c75e7d"}
Jul 24 03:49:09.532005 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.531937    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq" event={"ID":"2fa293c3-c15d-4e2e-9c9b-a2020fb25032","Type":"ContainerDied","Data":"9f740e5eafa957a48e7f2c5c11f7205134b3f870da84e6359f1e804aa9c49688"}
Jul 24 03:49:09.541651 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.541629    2574 scope.go:117] "RemoveContainer" containerID="19dc672da27d07ba066b7b210fa41ebcacd313c2b874171d852c26d44c23110b"
Jul 24 03:49:09.542032 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:49:09.541957    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19dc672da27d07ba066b7b210fa41ebcacd313c2b874171d852c26d44c23110b\": container with ID starting with 19dc672da27d07ba066b7b210fa41ebcacd313c2b874171d852c26d44c23110b not found: ID does not exist" containerID="19dc672da27d07ba066b7b210fa41ebcacd313c2b874171d852c26d44c23110b"
Jul 24 03:49:09.542032 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.541988    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19dc672da27d07ba066b7b210fa41ebcacd313c2b874171d852c26d44c23110b"} err="failed to get container status \"19dc672da27d07ba066b7b210fa41ebcacd313c2b874171d852c26d44c23110b\": rpc error: code = NotFound desc = could not find container \"19dc672da27d07ba066b7b210fa41ebcacd313c2b874171d852c26d44c23110b\": container with ID starting with 19dc672da27d07ba066b7b210fa41ebcacd313c2b874171d852c26d44c23110b not found: ID does not exist"
Jul 24 03:49:09.542032 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.542006    2574 scope.go:117] "RemoveContainer" containerID="1c8075f82d4590fa8594479955bed836da028b79f8bb98f0c129645b25c75e7d"
Jul 24 03:49:09.550552 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.550529    2574 scope.go:117] "RemoveContainer" containerID="1c8075f82d4590fa8594479955bed836da028b79f8bb98f0c129645b25c75e7d"
Jul 24 03:49:09.550837 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:49:09.550819    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c8075f82d4590fa8594479955bed836da028b79f8bb98f0c129645b25c75e7d\": container with ID starting with 1c8075f82d4590fa8594479955bed836da028b79f8bb98f0c129645b25c75e7d not found: ID does not exist" containerID="1c8075f82d4590fa8594479955bed836da028b79f8bb98f0c129645b25c75e7d"
Jul 24 03:49:09.550893 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.550846    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c8075f82d4590fa8594479955bed836da028b79f8bb98f0c129645b25c75e7d"} err="failed to get container status \"1c8075f82d4590fa8594479955bed836da028b79f8bb98f0c129645b25c75e7d\": rpc error: code = NotFound desc = could not find container \"1c8075f82d4590fa8594479955bed836da028b79f8bb98f0c129645b25c75e7d\": container with ID starting with 1c8075f82d4590fa8594479955bed836da028b79f8bb98f0c129645b25c75e7d not found: ID does not exist"
Jul 24 03:49:09.556029 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.556006    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"]
Jul 24 03:49:09.557778 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.557758    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-781d8b17-54749447d5-gpz6q"]
Jul 24 03:49:09.567092 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.567056    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"]
Jul 24 03:49:09.574351 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.574324    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b70af663-07c7-4d7f-a36d-faeb9b2eeeab" path="/var/lib/kubelet/pods/b70af663-07c7-4d7f-a36d-faeb9b2eeeab/volumes"
Jul 24 03:49:09.574641 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:09.574627    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-781d8b17-7d89845f9b-v5hpq"]
Jul 24 03:49:11.573070 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:11.573039    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fa293c3-c15d-4e2e-9c9b-a2020fb25032" path="/var/lib/kubelet/pods/2fa293c3-c15d-4e2e-9c9b-a2020fb25032/volumes"
Jul 24 03:49:13.464793 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:13.464756    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg"]
Jul 24 03:49:13.468320 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:13.468283    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-781d8b17-6zblg"]
Jul 24 03:49:13.573306 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:13.573266    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59f38a30-ac1d-47e8-b956-8c0f6488db81" path="/var/lib/kubelet/pods/59f38a30-ac1d-47e8-b956-8c0f6488db81/volumes"
Jul 24 03:49:17.122681 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.122644    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"]
Jul 24 03:49:17.123114 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.123042    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b70af663-07c7-4d7f-a36d-faeb9b2eeeab" containerName="payload-processing"
Jul 24 03:49:17.123114 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.123054    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b70af663-07c7-4d7f-a36d-faeb9b2eeeab" containerName="payload-processing"
Jul 24 03:49:17.123114 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.123063    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59f38a30-ac1d-47e8-b956-8c0f6488db81" containerName="revoke-keys"
Jul 24 03:49:17.123114 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.123068    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="59f38a30-ac1d-47e8-b956-8c0f6488db81" containerName="revoke-keys"
Jul 24 03:49:17.123114 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.123079    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fa293c3-c15d-4e2e-9c9b-a2020fb25032" containerName="payload-pre-processing"
Jul 24 03:49:17.123114 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.123084    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fa293c3-c15d-4e2e-9c9b-a2020fb25032" containerName="payload-pre-processing"
Jul 24 03:49:17.123330 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.123166    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fa293c3-c15d-4e2e-9c9b-a2020fb25032" containerName="payload-pre-processing"
Jul 24 03:49:17.123330 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.123177    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="59f38a30-ac1d-47e8-b956-8c0f6488db81" containerName="revoke-keys"
Jul 24 03:49:17.123330 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.123184    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b70af663-07c7-4d7f-a36d-faeb9b2eeeab" containerName="payload-processing"
Jul 24 03:49:17.127725 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.127706    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"
Jul 24 03:49:17.130139 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.130099    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-c4c112d8\""
Jul 24 03:49:17.137300 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.137273    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"]
Jul 24 03:49:17.162572 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.162536    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"]
Jul 24 03:49:17.165870 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.165850    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"
Jul 24 03:49:17.168704 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.168678    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-c4c112d8-dockercfg-vgrrh\""
Jul 24 03:49:17.177075 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.177047    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3baa5ba-b9f2-4431-a7c3-aa04134833c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb\" (UID: \"d3baa5ba-b9f2-4431-a7c3-aa04134833c8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"
Jul 24 03:49:17.177214 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.177087    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxr5b\" (UniqueName: \"kubernetes.io/projected/d3baa5ba-b9f2-4431-a7c3-aa04134833c8-kube-api-access-wxr5b\") pod \"payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb\" (UID: \"d3baa5ba-b9f2-4431-a7c3-aa04134833c8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"
Jul 24 03:49:17.177684 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.177667    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"]
Jul 24 03:49:17.277914 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.277877    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4ls7\" (UniqueName: \"kubernetes.io/projected/5b95b00a-a93f-4f68-8efa-e73559bf3a1d-kube-api-access-p4ls7\") pod \"payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj\" (UID: \"5b95b00a-a93f-4f68-8efa-e73559bf3a1d\") " pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"
Jul 24 03:49:17.278102 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.278002    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b95b00a-a93f-4f68-8efa-e73559bf3a1d-plugins-config-volume\") pod \"payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj\" (UID: \"5b95b00a-a93f-4f68-8efa-e73559bf3a1d\") " pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"
Jul 24 03:49:17.278102 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.278062    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3baa5ba-b9f2-4431-a7c3-aa04134833c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb\" (UID: \"d3baa5ba-b9f2-4431-a7c3-aa04134833c8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"
Jul 24 03:49:17.278102 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.278085    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxr5b\" (UniqueName: \"kubernetes.io/projected/d3baa5ba-b9f2-4431-a7c3-aa04134833c8-kube-api-access-wxr5b\") pod \"payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb\" (UID: \"d3baa5ba-b9f2-4431-a7c3-aa04134833c8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"
Jul 24 03:49:17.278767 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.278751    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3baa5ba-b9f2-4431-a7c3-aa04134833c8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb\" (UID: \"d3baa5ba-b9f2-4431-a7c3-aa04134833c8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"
Jul 24 03:49:17.287931 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.287899    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxr5b\" (UniqueName: \"kubernetes.io/projected/d3baa5ba-b9f2-4431-a7c3-aa04134833c8-kube-api-access-wxr5b\") pod \"payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb\" (UID: \"d3baa5ba-b9f2-4431-a7c3-aa04134833c8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"
Jul 24 03:49:17.379727 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.379633    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4ls7\" (UniqueName: \"kubernetes.io/projected/5b95b00a-a93f-4f68-8efa-e73559bf3a1d-kube-api-access-p4ls7\") pod \"payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj\" (UID: \"5b95b00a-a93f-4f68-8efa-e73559bf3a1d\") " pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"
Jul 24 03:49:17.379727 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.379705    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b95b00a-a93f-4f68-8efa-e73559bf3a1d-plugins-config-volume\") pod \"payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj\" (UID: \"5b95b00a-a93f-4f68-8efa-e73559bf3a1d\") " pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"
Jul 24 03:49:17.380365 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.380345    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b95b00a-a93f-4f68-8efa-e73559bf3a1d-plugins-config-volume\") pod \"payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj\" (UID: \"5b95b00a-a93f-4f68-8efa-e73559bf3a1d\") " pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"
Jul 24 03:49:17.388327 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.388299    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4ls7\" (UniqueName: \"kubernetes.io/projected/5b95b00a-a93f-4f68-8efa-e73559bf3a1d-kube-api-access-p4ls7\") pod \"payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj\" (UID: \"5b95b00a-a93f-4f68-8efa-e73559bf3a1d\") " pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"
Jul 24 03:49:17.440744 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.440693    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"
Jul 24 03:49:17.477818 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.477775    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"
Jul 24 03:49:17.612483 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.611617    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"]
Jul 24 03:49:17.614067 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:49:17.614031    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3baa5ba_b9f2_4431_a7c3_aa04134833c8.slice/crio-fa9e20b9c8c326bd608ad4fb7b14d647fedba0e3b6c868eca825363437622e27 WatchSource:0}: Error finding container fa9e20b9c8c326bd608ad4fb7b14d647fedba0e3b6c868eca825363437622e27: Status 404 returned error can't find the container with id fa9e20b9c8c326bd608ad4fb7b14d647fedba0e3b6c868eca825363437622e27
Jul 24 03:49:17.665694 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:17.665548    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"]
Jul 24 03:49:17.668614 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:49:17.668585    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b95b00a_a93f_4f68_8efa_e73559bf3a1d.slice/crio-2bc4092aeeda1132bf1221759a1ec7b976b32fc4e64ba9a42c6f7e8490b25228 WatchSource:0}: Error finding container 2bc4092aeeda1132bf1221759a1ec7b976b32fc4e64ba9a42c6f7e8490b25228: Status 404 returned error can't find the container with id 2bc4092aeeda1132bf1221759a1ec7b976b32fc4e64ba9a42c6f7e8490b25228
Jul 24 03:49:18.597004 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:18.596964    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj" event={"ID":"5b95b00a-a93f-4f68-8efa-e73559bf3a1d","Type":"ContainerStarted","Data":"fba31de4bb1e6ec3bdd404d5db8f8aad784ef8bd2bdf8a63293b3051526e7ea0"}
Jul 24 03:49:18.597487 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:18.597018    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj" event={"ID":"5b95b00a-a93f-4f68-8efa-e73559bf3a1d","Type":"ContainerStarted","Data":"2bc4092aeeda1132bf1221759a1ec7b976b32fc4e64ba9a42c6f7e8490b25228"}
Jul 24 03:49:18.597487 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:18.597045    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"
Jul 24 03:49:18.598703 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:18.598677    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb" event={"ID":"d3baa5ba-b9f2-4431-a7c3-aa04134833c8","Type":"ContainerStarted","Data":"5959a49071037a444a2ef2429b92b3477464df0eab145ebf78c5b9ee79dc8325"}
Jul 24 03:49:18.598830 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:18.598725    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb" event={"ID":"d3baa5ba-b9f2-4431-a7c3-aa04134833c8","Type":"ContainerStarted","Data":"fa9e20b9c8c326bd608ad4fb7b14d647fedba0e3b6c868eca825363437622e27"}
Jul 24 03:49:18.598830 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:18.598775    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"
Jul 24 03:49:18.613556 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:18.613510    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj" podStartSLOduration=1.613492816 podStartE2EDuration="1.613492816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:49:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:49:18.611371267 +0000 UTC m=+1913.574464555" watchObservedRunningTime="2026-07-24 03:49:18.613492816 +0000 UTC m=+1913.576586148"
Jul 24 03:49:18.625370 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:18.625314    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb" podStartSLOduration=1.625294494 podStartE2EDuration="1.625294494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:49:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:49:18.625283784 +0000 UTC m=+1913.588377071" watchObservedRunningTime="2026-07-24 03:49:18.625294494 +0000 UTC m=+1913.588387784"
Jul 24 03:49:29.605717 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:29.605664    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb" podUID="d3baa5ba-b9f2-4431-a7c3-aa04134833c8" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.42:9004: connect: connection refused"
Jul 24 03:49:29.606275 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:29.606111    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"
Jul 24 03:49:37.441625 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:37.441569    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb" podUID="d3baa5ba-b9f2-4431-a7c3-aa04134833c8" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.42:9004: connect: connection refused"
Jul 24 03:49:39.604402 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:39.604347    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb" podUID="d3baa5ba-b9f2-4431-a7c3-aa04134833c8" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.42:9004: connect: connection refused"
Jul 24 03:49:42.193815 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:42.193777    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n"]
Jul 24 03:49:42.204583 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:42.204552    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n"
Jul 24 03:49:42.205685 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:42.205653    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n"]
Jul 24 03:49:42.207142 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:42.207118    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 03:49:42.207317 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:42.207124    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 03:49:42.213139 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:42.213051    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e03c723-1f58-43d4-8ff8-94b9cd4a0073-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n\" (UID: \"8e03c723-1f58-43d4-8ff8-94b9cd4a0073\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n"
Jul 24 03:49:42.313805 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:42.313761    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e03c723-1f58-43d4-8ff8-94b9cd4a0073-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n\" (UID: \"8e03c723-1f58-43d4-8ff8-94b9cd4a0073\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n"
Jul 24 03:49:42.314365 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:42.314346    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e03c723-1f58-43d4-8ff8-94b9cd4a0073-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n\" (UID: \"8e03c723-1f58-43d4-8ff8-94b9cd4a0073\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n"
Jul 24 03:49:42.518245 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:42.518153    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n"
Jul 24 03:49:42.657102 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:42.657078    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n"]
Jul 24 03:49:42.659197 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:49:42.659170    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e03c723_1f58_43d4_8ff8_94b9cd4a0073.slice/crio-2443164be31f457dc56c0591961ea2dfc4e375b7e90152ae39f8bbb562fcd713 WatchSource:0}: Error finding container 2443164be31f457dc56c0591961ea2dfc4e375b7e90152ae39f8bbb562fcd713: Status 404 returned error can't find the container with id 2443164be31f457dc56c0591961ea2dfc4e375b7e90152ae39f8bbb562fcd713
Jul 24 03:49:42.719494 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:42.719456    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n" event={"ID":"8e03c723-1f58-43d4-8ff8-94b9cd4a0073","Type":"ContainerStarted","Data":"2443164be31f457dc56c0591961ea2dfc4e375b7e90152ae39f8bbb562fcd713"}
Jul 24 03:49:43.726548 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:43.726509    2574 generic.go:356] "Generic (PLEG): container finished" podID="8e03c723-1f58-43d4-8ff8-94b9cd4a0073" containerID="c473874911f677e27a73416a764e99d1f13974dd0fd64e0d4734c1d8bb49a2a7" exitCode=0
Jul 24 03:49:43.726995 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:43.726607    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n" event={"ID":"8e03c723-1f58-43d4-8ff8-94b9cd4a0073","Type":"ContainerDied","Data":"c473874911f677e27a73416a764e99d1f13974dd0fd64e0d4734c1d8bb49a2a7"}
Jul 24 03:49:44.865472 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:44.865446    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n"
Jul 24 03:49:44.936809 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:44.936775    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e03c723-1f58-43d4-8ff8-94b9cd4a0073-maas-api-service-ca\") pod \"8e03c723-1f58-43d4-8ff8-94b9cd4a0073\" (UID: \"8e03c723-1f58-43d4-8ff8-94b9cd4a0073\") "
Jul 24 03:49:44.937145 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:44.937121    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e03c723-1f58-43d4-8ff8-94b9cd4a0073-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8e03c723-1f58-43d4-8ff8-94b9cd4a0073" (UID: "8e03c723-1f58-43d4-8ff8-94b9cd4a0073"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:49:45.038591 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:45.038494    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e03c723-1f58-43d4-8ff8-94b9cd4a0073-maas-api-service-ca\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:49:45.738175 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:45.738141    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n"
Jul 24 03:49:45.738380 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:45.738192    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n" event={"ID":"8e03c723-1f58-43d4-8ff8-94b9cd4a0073","Type":"ContainerDied","Data":"2443164be31f457dc56c0591961ea2dfc4e375b7e90152ae39f8bbb562fcd713"}
Jul 24 03:49:45.738380 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:45.738231    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2443164be31f457dc56c0591961ea2dfc4e375b7e90152ae39f8bbb562fcd713"
Jul 24 03:49:49.605600 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:49.605568    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"
Jul 24 03:49:52.647501 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:52.647458    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"]
Jul 24 03:49:52.647952 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:52.647786    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj" podUID="5b95b00a-a93f-4f68-8efa-e73559bf3a1d" containerName="payload-processing" containerID="cri-o://fba31de4bb1e6ec3bdd404d5db8f8aad784ef8bd2bdf8a63293b3051526e7ea0" gracePeriod=30
Jul 24 03:49:52.658420 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:52.658370    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"]
Jul 24 03:49:52.658688 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:52.658625    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb" podUID="d3baa5ba-b9f2-4431-a7c3-aa04134833c8" containerName="payload-pre-processing" containerID="cri-o://5959a49071037a444a2ef2429b92b3477464df0eab145ebf78c5b9ee79dc8325" gracePeriod=30
Jul 24 03:49:52.774735 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:52.774704    2574 generic.go:356] "Generic (PLEG): container finished" podID="5b95b00a-a93f-4f68-8efa-e73559bf3a1d" containerID="fba31de4bb1e6ec3bdd404d5db8f8aad784ef8bd2bdf8a63293b3051526e7ea0" exitCode=0
Jul 24 03:49:52.774905 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:52.774748    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj" event={"ID":"5b95b00a-a93f-4f68-8efa-e73559bf3a1d","Type":"ContainerDied","Data":"fba31de4bb1e6ec3bdd404d5db8f8aad784ef8bd2bdf8a63293b3051526e7ea0"}
Jul 24 03:49:52.951023 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:52.950993    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"
Jul 24 03:49:52.954305 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:52.954281    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"
Jul 24 03:49:53.018693 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.018652    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b95b00a-a93f-4f68-8efa-e73559bf3a1d-plugins-config-volume\") pod \"5b95b00a-a93f-4f68-8efa-e73559bf3a1d\" (UID: \"5b95b00a-a93f-4f68-8efa-e73559bf3a1d\") "
Jul 24 03:49:53.018893 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.018761    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4ls7\" (UniqueName: \"kubernetes.io/projected/5b95b00a-a93f-4f68-8efa-e73559bf3a1d-kube-api-access-p4ls7\") pod \"5b95b00a-a93f-4f68-8efa-e73559bf3a1d\" (UID: \"5b95b00a-a93f-4f68-8efa-e73559bf3a1d\") "
Jul 24 03:49:53.018893 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.018796    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3baa5ba-b9f2-4431-a7c3-aa04134833c8-plugins-config-volume\") pod \"d3baa5ba-b9f2-4431-a7c3-aa04134833c8\" (UID: \"d3baa5ba-b9f2-4431-a7c3-aa04134833c8\") "
Jul 24 03:49:53.018893 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.018828    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxr5b\" (UniqueName: \"kubernetes.io/projected/d3baa5ba-b9f2-4431-a7c3-aa04134833c8-kube-api-access-wxr5b\") pod \"d3baa5ba-b9f2-4431-a7c3-aa04134833c8\" (UID: \"d3baa5ba-b9f2-4431-a7c3-aa04134833c8\") "
Jul 24 03:49:53.019147 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.019119    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b95b00a-a93f-4f68-8efa-e73559bf3a1d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b95b00a-a93f-4f68-8efa-e73559bf3a1d" (UID: "5b95b00a-a93f-4f68-8efa-e73559bf3a1d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:49:53.019206 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.019185    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3baa5ba-b9f2-4431-a7c3-aa04134833c8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d3baa5ba-b9f2-4431-a7c3-aa04134833c8" (UID: "d3baa5ba-b9f2-4431-a7c3-aa04134833c8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:49:53.021073 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.021048    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3baa5ba-b9f2-4431-a7c3-aa04134833c8-kube-api-access-wxr5b" (OuterVolumeSpecName: "kube-api-access-wxr5b") pod "d3baa5ba-b9f2-4431-a7c3-aa04134833c8" (UID: "d3baa5ba-b9f2-4431-a7c3-aa04134833c8"). InnerVolumeSpecName "kube-api-access-wxr5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:49:53.021235 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.021209    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b95b00a-a93f-4f68-8efa-e73559bf3a1d-kube-api-access-p4ls7" (OuterVolumeSpecName: "kube-api-access-p4ls7") pod "5b95b00a-a93f-4f68-8efa-e73559bf3a1d" (UID: "5b95b00a-a93f-4f68-8efa-e73559bf3a1d"). InnerVolumeSpecName "kube-api-access-p4ls7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:49:53.119858 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.119806    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b95b00a-a93f-4f68-8efa-e73559bf3a1d-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:49:53.119858 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.119849    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4ls7\" (UniqueName: \"kubernetes.io/projected/5b95b00a-a93f-4f68-8efa-e73559bf3a1d-kube-api-access-p4ls7\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:49:53.119858 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.119859    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3baa5ba-b9f2-4431-a7c3-aa04134833c8-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:49:53.119858 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.119867    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxr5b\" (UniqueName: \"kubernetes.io/projected/d3baa5ba-b9f2-4431-a7c3-aa04134833c8-kube-api-access-wxr5b\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:49:53.781383 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.781349    2574 generic.go:356] "Generic (PLEG): container finished" podID="d3baa5ba-b9f2-4431-a7c3-aa04134833c8" containerID="5959a49071037a444a2ef2429b92b3477464df0eab145ebf78c5b9ee79dc8325" exitCode=0
Jul 24 03:49:53.781838 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.781395    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"
Jul 24 03:49:53.781838 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.781470    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb" event={"ID":"d3baa5ba-b9f2-4431-a7c3-aa04134833c8","Type":"ContainerDied","Data":"5959a49071037a444a2ef2429b92b3477464df0eab145ebf78c5b9ee79dc8325"}
Jul 24 03:49:53.781838 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.781519    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb" event={"ID":"d3baa5ba-b9f2-4431-a7c3-aa04134833c8","Type":"ContainerDied","Data":"fa9e20b9c8c326bd608ad4fb7b14d647fedba0e3b6c868eca825363437622e27"}
Jul 24 03:49:53.781838 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.781544    2574 scope.go:117] "RemoveContainer" containerID="5959a49071037a444a2ef2429b92b3477464df0eab145ebf78c5b9ee79dc8325"
Jul 24 03:49:53.783015 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.782985    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj" event={"ID":"5b95b00a-a93f-4f68-8efa-e73559bf3a1d","Type":"ContainerDied","Data":"2bc4092aeeda1132bf1221759a1ec7b976b32fc4e64ba9a42c6f7e8490b25228"}
Jul 24 03:49:53.783015 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.783000    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"
Jul 24 03:49:53.791793 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.791767    2574 scope.go:117] "RemoveContainer" containerID="5959a49071037a444a2ef2429b92b3477464df0eab145ebf78c5b9ee79dc8325"
Jul 24 03:49:53.792157 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:49:53.792104    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5959a49071037a444a2ef2429b92b3477464df0eab145ebf78c5b9ee79dc8325\": container with ID starting with 5959a49071037a444a2ef2429b92b3477464df0eab145ebf78c5b9ee79dc8325 not found: ID does not exist" containerID="5959a49071037a444a2ef2429b92b3477464df0eab145ebf78c5b9ee79dc8325"
Jul 24 03:49:53.792281 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.792145    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5959a49071037a444a2ef2429b92b3477464df0eab145ebf78c5b9ee79dc8325"} err="failed to get container status \"5959a49071037a444a2ef2429b92b3477464df0eab145ebf78c5b9ee79dc8325\": rpc error: code = NotFound desc = could not find container \"5959a49071037a444a2ef2429b92b3477464df0eab145ebf78c5b9ee79dc8325\": container with ID starting with 5959a49071037a444a2ef2429b92b3477464df0eab145ebf78c5b9ee79dc8325 not found: ID does not exist"
Jul 24 03:49:53.792281 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.792181    2574 scope.go:117] "RemoveContainer" containerID="fba31de4bb1e6ec3bdd404d5db8f8aad784ef8bd2bdf8a63293b3051526e7ea0"
Jul 24 03:49:53.801528 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.801503    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"]
Jul 24 03:49:53.805309 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.805282    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c4c112d8-68ffbf48bd-v4btb"]
Jul 24 03:49:53.815035 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.815013    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"]
Jul 24 03:49:53.818558 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:53.818537    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c4c112d8-586656d744-kqhqj"]
Jul 24 03:49:55.573785 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:55.573743    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b95b00a-a93f-4f68-8efa-e73559bf3a1d" path="/var/lib/kubelet/pods/5b95b00a-a93f-4f68-8efa-e73559bf3a1d/volumes"
Jul 24 03:49:55.574288 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:55.574267    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3baa5ba-b9f2-4431-a7c3-aa04134833c8" path="/var/lib/kubelet/pods/d3baa5ba-b9f2-4431-a7c3-aa04134833c8/volumes"
Jul 24 03:49:57.275494 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:57.275448    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n"]
Jul 24 03:49:57.277804 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:57.277777    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c4c112d8-mm96n"]
Jul 24 03:49:57.573641 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:49:57.573557    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e03c723-1f58-43d4-8ff8-94b9cd4a0073" path="/var/lib/kubelet/pods/8e03c723-1f58-43d4-8ff8-94b9cd4a0073/volumes"
Jul 24 03:50:02.544067 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.544026    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"]
Jul 24 03:50:02.544584 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.544563    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e03c723-1f58-43d4-8ff8-94b9cd4a0073" containerName="revoke-keys"
Jul 24 03:50:02.544584 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.544577    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e03c723-1f58-43d4-8ff8-94b9cd4a0073" containerName="revoke-keys"
Jul 24 03:50:02.544660 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.544602    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3baa5ba-b9f2-4431-a7c3-aa04134833c8" containerName="payload-pre-processing"
Jul 24 03:50:02.544660 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.544609    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3baa5ba-b9f2-4431-a7c3-aa04134833c8" containerName="payload-pre-processing"
Jul 24 03:50:02.544660 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.544615    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b95b00a-a93f-4f68-8efa-e73559bf3a1d" containerName="payload-processing"
Jul 24 03:50:02.544660 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.544621    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b95b00a-a93f-4f68-8efa-e73559bf3a1d" containerName="payload-processing"
Jul 24 03:50:02.544794 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.544679    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e03c723-1f58-43d4-8ff8-94b9cd4a0073" containerName="revoke-keys"
Jul 24 03:50:02.544794 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.544688    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3baa5ba-b9f2-4431-a7c3-aa04134833c8" containerName="payload-pre-processing"
Jul 24 03:50:02.544794 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.544694    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b95b00a-a93f-4f68-8efa-e73559bf3a1d" containerName="payload-processing"
Jul 24 03:50:02.547597 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.547577    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"
Jul 24 03:50:02.550658 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.550634    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-0cd07409-dockercfg-x5z42\""
Jul 24 03:50:02.550901 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.550883    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-0cd07409\""
Jul 24 03:50:02.559803 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.559779    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"]
Jul 24 03:50:02.563668 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.563644    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"]
Jul 24 03:50:02.563789 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.563760    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"
Jul 24 03:50:02.578016 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.577986    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"]
Jul 24 03:50:02.611845 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.611803    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5\" (UID: \"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"
Jul 24 03:50:02.612024 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.611855    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm6hk\" (UniqueName: \"kubernetes.io/projected/4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7-kube-api-access-gm6hk\") pod \"payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5\" (UID: \"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"
Jul 24 03:50:02.712899 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.712863    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fmgq\" (UniqueName: \"kubernetes.io/projected/ae15e63e-ee43-4708-b026-4ad9d2d6ba12-kube-api-access-2fmgq\") pod \"payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5\" (UID: \"ae15e63e-ee43-4708-b026-4ad9d2d6ba12\") " pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"
Jul 24 03:50:02.713091 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.712911    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5\" (UID: \"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"
Jul 24 03:50:02.713091 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.712934    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae15e63e-ee43-4708-b026-4ad9d2d6ba12-plugins-config-volume\") pod \"payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5\" (UID: \"ae15e63e-ee43-4708-b026-4ad9d2d6ba12\") " pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"
Jul 24 03:50:02.713091 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.712989    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm6hk\" (UniqueName: \"kubernetes.io/projected/4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7-kube-api-access-gm6hk\") pod \"payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5\" (UID: \"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"
Jul 24 03:50:02.713641 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.713620    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5\" (UID: \"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"
Jul 24 03:50:02.722090 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.722058    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm6hk\" (UniqueName: \"kubernetes.io/projected/4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7-kube-api-access-gm6hk\") pod \"payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5\" (UID: \"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"
Jul 24 03:50:02.814228 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.814132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fmgq\" (UniqueName: \"kubernetes.io/projected/ae15e63e-ee43-4708-b026-4ad9d2d6ba12-kube-api-access-2fmgq\") pod \"payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5\" (UID: \"ae15e63e-ee43-4708-b026-4ad9d2d6ba12\") " pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"
Jul 24 03:50:02.814228 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.814187    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae15e63e-ee43-4708-b026-4ad9d2d6ba12-plugins-config-volume\") pod \"payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5\" (UID: \"ae15e63e-ee43-4708-b026-4ad9d2d6ba12\") " pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"
Jul 24 03:50:02.814939 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.814917    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae15e63e-ee43-4708-b026-4ad9d2d6ba12-plugins-config-volume\") pod \"payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5\" (UID: \"ae15e63e-ee43-4708-b026-4ad9d2d6ba12\") " pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"
Jul 24 03:50:02.822388 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.822363    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fmgq\" (UniqueName: \"kubernetes.io/projected/ae15e63e-ee43-4708-b026-4ad9d2d6ba12-kube-api-access-2fmgq\") pod \"payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5\" (UID: \"ae15e63e-ee43-4708-b026-4ad9d2d6ba12\") " pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"
Jul 24 03:50:02.858616 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.858578    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"
Jul 24 03:50:02.874590 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:02.874550    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"
Jul 24 03:50:03.020463 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:03.020419    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"]
Jul 24 03:50:03.022748 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:50:03.022718    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f9b56a8_09b0_4e98_a4f8_4119a7ca47f7.slice/crio-b2ac1502ace873a93cf28efa17508a4f0cad3165c8d74a286381f1436030f030 WatchSource:0}: Error finding container b2ac1502ace873a93cf28efa17508a4f0cad3165c8d74a286381f1436030f030: Status 404 returned error can't find the container with id b2ac1502ace873a93cf28efa17508a4f0cad3165c8d74a286381f1436030f030
Jul 24 03:50:03.055528 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:03.055504    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"]
Jul 24 03:50:03.057546 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:50:03.057518    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae15e63e_ee43_4708_b026_4ad9d2d6ba12.slice/crio-d9c878d5e5fd68e3d396a70f3c7bdc1266f2c890c9360a0fe2bb7139888b2476 WatchSource:0}: Error finding container d9c878d5e5fd68e3d396a70f3c7bdc1266f2c890c9360a0fe2bb7139888b2476: Status 404 returned error can't find the container with id d9c878d5e5fd68e3d396a70f3c7bdc1266f2c890c9360a0fe2bb7139888b2476
Jul 24 03:50:03.836686 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:03.836641    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5" event={"ID":"ae15e63e-ee43-4708-b026-4ad9d2d6ba12","Type":"ContainerStarted","Data":"720f639c71b29374bfbc5eabe3d9ee8c0f42ee00c9f293730de98859a3fcbbb8"}
Jul 24 03:50:03.837125 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:03.836709    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"
Jul 24 03:50:03.837125 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:03.836731    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5" event={"ID":"ae15e63e-ee43-4708-b026-4ad9d2d6ba12","Type":"ContainerStarted","Data":"d9c878d5e5fd68e3d396a70f3c7bdc1266f2c890c9360a0fe2bb7139888b2476"}
Jul 24 03:50:03.838537 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:03.838507    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5" event={"ID":"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7","Type":"ContainerStarted","Data":"11147ca491b5088a6f7b81266b59b5033c27de05e6e340f162914047f6583ff8"}
Jul 24 03:50:03.838669 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:03.838556    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5" event={"ID":"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7","Type":"ContainerStarted","Data":"b2ac1502ace873a93cf28efa17508a4f0cad3165c8d74a286381f1436030f030"}
Jul 24 03:50:03.838669 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:03.838645    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"
Jul 24 03:50:03.854442 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:03.854360    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5" podStartSLOduration=1.854338406 podStartE2EDuration="1.854338406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:50:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:50:03.852567449 +0000 UTC m=+1958.815660747" watchObservedRunningTime="2026-07-24 03:50:03.854338406 +0000 UTC m=+1958.817431694"
Jul 24 03:50:03.867242 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:03.867182    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5" podStartSLOduration=1.8671647249999999 podStartE2EDuration="1.867164725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:50:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:50:03.866832058 +0000 UTC m=+1958.829925338" watchObservedRunningTime="2026-07-24 03:50:03.867164725 +0000 UTC m=+1958.830258013"
Jul 24 03:50:14.845952 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:14.845921    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"
Jul 24 03:50:14.846412 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:14.845989    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"
Jul 24 03:50:26.483167 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:26.483124    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b"]
Jul 24 03:50:26.488055 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:26.488036    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b"
Jul 24 03:50:26.490364 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:26.490341    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 03:50:26.490542 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:26.490457    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 03:50:26.493942 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:26.493918    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b"]
Jul 24 03:50:26.537838 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:26.537804    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1065a342-7db6-4dfe-9093-ba8dc55eb4d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0cd07409-xq75b\" (UID: \"1065a342-7db6-4dfe-9093-ba8dc55eb4d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b"
Jul 24 03:50:26.638924 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:26.638879    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1065a342-7db6-4dfe-9093-ba8dc55eb4d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0cd07409-xq75b\" (UID: \"1065a342-7db6-4dfe-9093-ba8dc55eb4d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b"
Jul 24 03:50:26.639743 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:26.639724    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1065a342-7db6-4dfe-9093-ba8dc55eb4d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-0cd07409-xq75b\" (UID: \"1065a342-7db6-4dfe-9093-ba8dc55eb4d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b"
Jul 24 03:50:26.802568 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:26.802479    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b"
Jul 24 03:50:26.956458 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:26.956416    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b"]
Jul 24 03:50:26.958671 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:50:26.958637    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1065a342_7db6_4dfe_9093_ba8dc55eb4d8.slice/crio-b7f6b364dee9e4cf0d3e251ca04ef33aa7e35b0df7f63a2bef27d342e5bcdfa1 WatchSource:0}: Error finding container b7f6b364dee9e4cf0d3e251ca04ef33aa7e35b0df7f63a2bef27d342e5bcdfa1: Status 404 returned error can't find the container with id b7f6b364dee9e4cf0d3e251ca04ef33aa7e35b0df7f63a2bef27d342e5bcdfa1
Jul 24 03:50:27.964220 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:27.964126    2574 generic.go:356] "Generic (PLEG): container finished" podID="1065a342-7db6-4dfe-9093-ba8dc55eb4d8" containerID="7eb050b2907ac2dc88cec0ee1ac845af07a0f23b89adac0abedf0042efd4f479" exitCode=0
Jul 24 03:50:27.964220 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:27.964182    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b" event={"ID":"1065a342-7db6-4dfe-9093-ba8dc55eb4d8","Type":"ContainerDied","Data":"7eb050b2907ac2dc88cec0ee1ac845af07a0f23b89adac0abedf0042efd4f479"}
Jul 24 03:50:27.964649 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:27.964225    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b" event={"ID":"1065a342-7db6-4dfe-9093-ba8dc55eb4d8","Type":"ContainerStarted","Data":"b7f6b364dee9e4cf0d3e251ca04ef33aa7e35b0df7f63a2bef27d342e5bcdfa1"}
Jul 24 03:50:29.102513 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:29.102485    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b"
Jul 24 03:50:29.162356 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:29.162327    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1065a342-7db6-4dfe-9093-ba8dc55eb4d8-maas-api-service-ca\") pod \"1065a342-7db6-4dfe-9093-ba8dc55eb4d8\" (UID: \"1065a342-7db6-4dfe-9093-ba8dc55eb4d8\") "
Jul 24 03:50:29.162751 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:29.162726    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1065a342-7db6-4dfe-9093-ba8dc55eb4d8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1065a342-7db6-4dfe-9093-ba8dc55eb4d8" (UID: "1065a342-7db6-4dfe-9093-ba8dc55eb4d8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:50:29.263717 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:29.263630    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1065a342-7db6-4dfe-9093-ba8dc55eb4d8-maas-api-service-ca\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:50:29.975728 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:29.975694    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b"
Jul 24 03:50:29.975909 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:29.975730    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b" event={"ID":"1065a342-7db6-4dfe-9093-ba8dc55eb4d8","Type":"ContainerDied","Data":"b7f6b364dee9e4cf0d3e251ca04ef33aa7e35b0df7f63a2bef27d342e5bcdfa1"}
Jul 24 03:50:29.975909 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:29.975794    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7f6b364dee9e4cf0d3e251ca04ef33aa7e35b0df7f63a2bef27d342e5bcdfa1"
Jul 24 03:50:37.063100 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.063065    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"]
Jul 24 03:50:37.063821 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.063316    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5" podUID="ae15e63e-ee43-4708-b026-4ad9d2d6ba12" containerName="payload-processing" containerID="cri-o://720f639c71b29374bfbc5eabe3d9ee8c0f42ee00c9f293730de98859a3fcbbb8" gracePeriod=30
Jul 24 03:50:37.069888 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.069846    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"]
Jul 24 03:50:37.070107 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.070079    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5" podUID="4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7" containerName="payload-pre-processing" containerID="cri-o://11147ca491b5088a6f7b81266b59b5033c27de05e6e340f162914047f6583ff8" gracePeriod=30
Jul 24 03:50:37.352927 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.352901    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"
Jul 24 03:50:37.355811 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.355785    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"
Jul 24 03:50:37.437821 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.437791    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae15e63e-ee43-4708-b026-4ad9d2d6ba12-plugins-config-volume\") pod \"ae15e63e-ee43-4708-b026-4ad9d2d6ba12\" (UID: \"ae15e63e-ee43-4708-b026-4ad9d2d6ba12\") "
Jul 24 03:50:37.438010 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.437887    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fmgq\" (UniqueName: \"kubernetes.io/projected/ae15e63e-ee43-4708-b026-4ad9d2d6ba12-kube-api-access-2fmgq\") pod \"ae15e63e-ee43-4708-b026-4ad9d2d6ba12\" (UID: \"ae15e63e-ee43-4708-b026-4ad9d2d6ba12\") "
Jul 24 03:50:37.438010 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.437935    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm6hk\" (UniqueName: \"kubernetes.io/projected/4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7-kube-api-access-gm6hk\") pod \"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7\" (UID: \"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7\") "
Jul 24 03:50:37.438010 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.437970    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7-plugins-config-volume\") pod \"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7\" (UID: \"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7\") "
Jul 24 03:50:37.438327 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.438219    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae15e63e-ee43-4708-b026-4ad9d2d6ba12-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ae15e63e-ee43-4708-b026-4ad9d2d6ba12" (UID: "ae15e63e-ee43-4708-b026-4ad9d2d6ba12"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:50:37.438447 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.438367    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7" (UID: "4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:50:37.440189 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.440160    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7-kube-api-access-gm6hk" (OuterVolumeSpecName: "kube-api-access-gm6hk") pod "4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7" (UID: "4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7"). InnerVolumeSpecName "kube-api-access-gm6hk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:50:37.440291 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.440260    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae15e63e-ee43-4708-b026-4ad9d2d6ba12-kube-api-access-2fmgq" (OuterVolumeSpecName: "kube-api-access-2fmgq") pod "ae15e63e-ee43-4708-b026-4ad9d2d6ba12" (UID: "ae15e63e-ee43-4708-b026-4ad9d2d6ba12"). InnerVolumeSpecName "kube-api-access-2fmgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:50:37.538991 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.538945    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae15e63e-ee43-4708-b026-4ad9d2d6ba12-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:50:37.538991 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.538980    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fmgq\" (UniqueName: \"kubernetes.io/projected/ae15e63e-ee43-4708-b026-4ad9d2d6ba12-kube-api-access-2fmgq\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:50:37.538991 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.538995    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm6hk\" (UniqueName: \"kubernetes.io/projected/4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7-kube-api-access-gm6hk\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:50:37.539226 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:37.539006    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:50:38.017853 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.017821    2574 generic.go:356] "Generic (PLEG): container finished" podID="ae15e63e-ee43-4708-b026-4ad9d2d6ba12" containerID="720f639c71b29374bfbc5eabe3d9ee8c0f42ee00c9f293730de98859a3fcbbb8" exitCode=0
Jul 24 03:50:38.018060 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.017869    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"
Jul 24 03:50:38.018060 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.017910    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5" event={"ID":"ae15e63e-ee43-4708-b026-4ad9d2d6ba12","Type":"ContainerDied","Data":"720f639c71b29374bfbc5eabe3d9ee8c0f42ee00c9f293730de98859a3fcbbb8"}
Jul 24 03:50:38.018060 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.017962    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5" event={"ID":"ae15e63e-ee43-4708-b026-4ad9d2d6ba12","Type":"ContainerDied","Data":"d9c878d5e5fd68e3d396a70f3c7bdc1266f2c890c9360a0fe2bb7139888b2476"}
Jul 24 03:50:38.018060 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.017985    2574 scope.go:117] "RemoveContainer" containerID="720f639c71b29374bfbc5eabe3d9ee8c0f42ee00c9f293730de98859a3fcbbb8"
Jul 24 03:50:38.019336 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.019314    2574 generic.go:356] "Generic (PLEG): container finished" podID="4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7" containerID="11147ca491b5088a6f7b81266b59b5033c27de05e6e340f162914047f6583ff8" exitCode=0
Jul 24 03:50:38.019502 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.019360    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"
Jul 24 03:50:38.019502 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.019395    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5" event={"ID":"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7","Type":"ContainerDied","Data":"11147ca491b5088a6f7b81266b59b5033c27de05e6e340f162914047f6583ff8"}
Jul 24 03:50:38.019502 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.019453    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5" event={"ID":"4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7","Type":"ContainerDied","Data":"b2ac1502ace873a93cf28efa17508a4f0cad3165c8d74a286381f1436030f030"}
Jul 24 03:50:38.027166 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.027145    2574 scope.go:117] "RemoveContainer" containerID="720f639c71b29374bfbc5eabe3d9ee8c0f42ee00c9f293730de98859a3fcbbb8"
Jul 24 03:50:38.027497 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:50:38.027407    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"720f639c71b29374bfbc5eabe3d9ee8c0f42ee00c9f293730de98859a3fcbbb8\": container with ID starting with 720f639c71b29374bfbc5eabe3d9ee8c0f42ee00c9f293730de98859a3fcbbb8 not found: ID does not exist" containerID="720f639c71b29374bfbc5eabe3d9ee8c0f42ee00c9f293730de98859a3fcbbb8"
Jul 24 03:50:38.027497 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.027468    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"720f639c71b29374bfbc5eabe3d9ee8c0f42ee00c9f293730de98859a3fcbbb8"} err="failed to get container status \"720f639c71b29374bfbc5eabe3d9ee8c0f42ee00c9f293730de98859a3fcbbb8\": rpc error: code = NotFound desc = could not find container \"720f639c71b29374bfbc5eabe3d9ee8c0f42ee00c9f293730de98859a3fcbbb8\": container with ID starting with 720f639c71b29374bfbc5eabe3d9ee8c0f42ee00c9f293730de98859a3fcbbb8 not found: ID does not exist"
Jul 24 03:50:38.027497 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.027489    2574 scope.go:117] "RemoveContainer" containerID="11147ca491b5088a6f7b81266b59b5033c27de05e6e340f162914047f6583ff8"
Jul 24 03:50:38.036174 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.036155    2574 scope.go:117] "RemoveContainer" containerID="11147ca491b5088a6f7b81266b59b5033c27de05e6e340f162914047f6583ff8"
Jul 24 03:50:38.036509 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:50:38.036485    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11147ca491b5088a6f7b81266b59b5033c27de05e6e340f162914047f6583ff8\": container with ID starting with 11147ca491b5088a6f7b81266b59b5033c27de05e6e340f162914047f6583ff8 not found: ID does not exist" containerID="11147ca491b5088a6f7b81266b59b5033c27de05e6e340f162914047f6583ff8"
Jul 24 03:50:38.036582 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.036522    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11147ca491b5088a6f7b81266b59b5033c27de05e6e340f162914047f6583ff8"} err="failed to get container status \"11147ca491b5088a6f7b81266b59b5033c27de05e6e340f162914047f6583ff8\": rpc error: code = NotFound desc = could not find container \"11147ca491b5088a6f7b81266b59b5033c27de05e6e340f162914047f6583ff8\": container with ID starting with 11147ca491b5088a6f7b81266b59b5033c27de05e6e340f162914047f6583ff8 not found: ID does not exist"
Jul 24 03:50:38.037021 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.037003    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"]
Jul 24 03:50:38.040569 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.040547    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0cd07409-6695bf674d-rfkr5"]
Jul 24 03:50:38.054309 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.054270    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"]
Jul 24 03:50:38.056125 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:38.056097    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0cd07409-547c7f5b8b-986h5"]
Jul 24 03:50:39.572945 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:39.572908    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7" path="/var/lib/kubelet/pods/4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7/volumes"
Jul 24 03:50:39.573295 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:39.573242    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae15e63e-ee43-4708-b026-4ad9d2d6ba12" path="/var/lib/kubelet/pods/ae15e63e-ee43-4708-b026-4ad9d2d6ba12/volumes"
Jul 24 03:50:41.556728 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:41.556698    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b"]
Jul 24 03:50:41.561686 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:41.561656    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-0cd07409-xq75b"]
Jul 24 03:50:41.572730 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:50:41.572704    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1065a342-7db6-4dfe-9093-ba8dc55eb4d8" path="/var/lib/kubelet/pods/1065a342-7db6-4dfe-9093-ba8dc55eb4d8/volumes"
Jul 24 03:52:23.935898 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.935859    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"]
Jul 24 03:52:23.936519 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.936421    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7" containerName="payload-pre-processing"
Jul 24 03:52:23.936519 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.936456    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7" containerName="payload-pre-processing"
Jul 24 03:52:23.936519 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.936480    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae15e63e-ee43-4708-b026-4ad9d2d6ba12" containerName="payload-processing"
Jul 24 03:52:23.936519 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.936490    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae15e63e-ee43-4708-b026-4ad9d2d6ba12" containerName="payload-processing"
Jul 24 03:52:23.936519 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.936519    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1065a342-7db6-4dfe-9093-ba8dc55eb4d8" containerName="revoke-keys"
Jul 24 03:52:23.936973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.936529    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1065a342-7db6-4dfe-9093-ba8dc55eb4d8" containerName="revoke-keys"
Jul 24 03:52:23.936973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.936610    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f9b56a8-09b0-4e98-a4f8-4119a7ca47f7" containerName="payload-pre-processing"
Jul 24 03:52:23.936973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.936626    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1065a342-7db6-4dfe-9093-ba8dc55eb4d8" containerName="revoke-keys"
Jul 24 03:52:23.936973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.936636    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae15e63e-ee43-4708-b026-4ad9d2d6ba12" containerName="payload-processing"
Jul 24 03:52:23.940300 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.940274    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"
Jul 24 03:52:23.942759 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.942708    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-365be7db\""
Jul 24 03:52:23.942916 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.942823    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-365be7db-dockercfg-jvdtn\""
Jul 24 03:52:23.950171 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.949772    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"]
Jul 24 03:52:23.965202 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.965167    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"]
Jul 24 03:52:23.968842 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.968822    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"
Jul 24 03:52:23.978267 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:23.978238    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"]
Jul 24 03:52:24.058624 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.058591    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjlmb\" (UniqueName: \"kubernetes.io/projected/ece4aa0b-66e5-4d9e-8992-64b472a0378c-kube-api-access-bjlmb\") pod \"payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2\" (UID: \"ece4aa0b-66e5-4d9e-8992-64b472a0378c\") " pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"
Jul 24 03:52:24.058624 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.058628    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ece4aa0b-66e5-4d9e-8992-64b472a0378c-plugins-config-volume\") pod \"payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2\" (UID: \"ece4aa0b-66e5-4d9e-8992-64b472a0378c\") " pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"
Jul 24 03:52:24.058866 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.058658    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45373c2d-6207-4c1d-99ba-15cf810b3cb3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h\" (UID: \"45373c2d-6207-4c1d-99ba-15cf810b3cb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"
Jul 24 03:52:24.058866 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.058762    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9ttx\" (UniqueName: \"kubernetes.io/projected/45373c2d-6207-4c1d-99ba-15cf810b3cb3-kube-api-access-v9ttx\") pod \"payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h\" (UID: \"45373c2d-6207-4c1d-99ba-15cf810b3cb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"
Jul 24 03:52:24.159773 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.159731    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45373c2d-6207-4c1d-99ba-15cf810b3cb3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h\" (UID: \"45373c2d-6207-4c1d-99ba-15cf810b3cb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"
Jul 24 03:52:24.159987 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.159814    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9ttx\" (UniqueName: \"kubernetes.io/projected/45373c2d-6207-4c1d-99ba-15cf810b3cb3-kube-api-access-v9ttx\") pod \"payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h\" (UID: \"45373c2d-6207-4c1d-99ba-15cf810b3cb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"
Jul 24 03:52:24.159987 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.159864    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjlmb\" (UniqueName: \"kubernetes.io/projected/ece4aa0b-66e5-4d9e-8992-64b472a0378c-kube-api-access-bjlmb\") pod \"payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2\" (UID: \"ece4aa0b-66e5-4d9e-8992-64b472a0378c\") " pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"
Jul 24 03:52:24.159987 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.159882    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ece4aa0b-66e5-4d9e-8992-64b472a0378c-plugins-config-volume\") pod \"payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2\" (UID: \"ece4aa0b-66e5-4d9e-8992-64b472a0378c\") " pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"
Jul 24 03:52:24.160500 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.160478    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45373c2d-6207-4c1d-99ba-15cf810b3cb3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h\" (UID: \"45373c2d-6207-4c1d-99ba-15cf810b3cb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"
Jul 24 03:52:24.160574 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.160549    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ece4aa0b-66e5-4d9e-8992-64b472a0378c-plugins-config-volume\") pod \"payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2\" (UID: \"ece4aa0b-66e5-4d9e-8992-64b472a0378c\") " pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"
Jul 24 03:52:24.169068 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.169032    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjlmb\" (UniqueName: \"kubernetes.io/projected/ece4aa0b-66e5-4d9e-8992-64b472a0378c-kube-api-access-bjlmb\") pod \"payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2\" (UID: \"ece4aa0b-66e5-4d9e-8992-64b472a0378c\") " pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"
Jul 24 03:52:24.169178 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.169069    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9ttx\" (UniqueName: \"kubernetes.io/projected/45373c2d-6207-4c1d-99ba-15cf810b3cb3-kube-api-access-v9ttx\") pod \"payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h\" (UID: \"45373c2d-6207-4c1d-99ba-15cf810b3cb3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"
Jul 24 03:52:24.254745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.254649    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"
Jul 24 03:52:24.280290 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.280262    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"
Jul 24 03:52:24.414106 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.414046    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"]
Jul 24 03:52:24.417400 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:52:24.417365    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45373c2d_6207_4c1d_99ba_15cf810b3cb3.slice/crio-8d404fa6a7e5420052470ec88a91edca209cc412030fc2d520732544534f70bc WatchSource:0}: Error finding container 8d404fa6a7e5420052470ec88a91edca209cc412030fc2d520732544534f70bc: Status 404 returned error can't find the container with id 8d404fa6a7e5420052470ec88a91edca209cc412030fc2d520732544534f70bc
Jul 24 03:52:24.448083 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.448042    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"]
Jul 24 03:52:24.452765 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:52:24.452737    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece4aa0b_66e5_4d9e_8992_64b472a0378c.slice/crio-86da213fcb6c94ee9b01bdd747b4a9c163ee104e19bf12707ec1a31275b6b0d8 WatchSource:0}: Error finding container 86da213fcb6c94ee9b01bdd747b4a9c163ee104e19bf12707ec1a31275b6b0d8: Status 404 returned error can't find the container with id 86da213fcb6c94ee9b01bdd747b4a9c163ee104e19bf12707ec1a31275b6b0d8
Jul 24 03:52:24.551258 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.551163    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h" event={"ID":"45373c2d-6207-4c1d-99ba-15cf810b3cb3","Type":"ContainerStarted","Data":"026c7fd7270d8fb6e596af1baecd18ab2cd5469f9d8af7fa5f8679b5a60b6d4c"}
Jul 24 03:52:24.551258 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.551217    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h" event={"ID":"45373c2d-6207-4c1d-99ba-15cf810b3cb3","Type":"ContainerStarted","Data":"8d404fa6a7e5420052470ec88a91edca209cc412030fc2d520732544534f70bc"}
Jul 24 03:52:24.551533 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.551269    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"
Jul 24 03:52:24.553008 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.552979    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2" event={"ID":"ece4aa0b-66e5-4d9e-8992-64b472a0378c","Type":"ContainerStarted","Data":"24bf4b72ef92c7ec391599e302a377359ff78c7f6eae5c5379ba062622132450"}
Jul 24 03:52:24.553158 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.553026    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2" event={"ID":"ece4aa0b-66e5-4d9e-8992-64b472a0378c","Type":"ContainerStarted","Data":"86da213fcb6c94ee9b01bdd747b4a9c163ee104e19bf12707ec1a31275b6b0d8"}
Jul 24 03:52:24.553158 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.553106    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"
Jul 24 03:52:24.567737 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.567682    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h" podStartSLOduration=1.567664369 podStartE2EDuration="1.567664369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:52:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:52:24.566753519 +0000 UTC m=+2099.529846818" watchObservedRunningTime="2026-07-24 03:52:24.567664369 +0000 UTC m=+2099.530757656"
Jul 24 03:52:24.583729 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:24.583676    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2" podStartSLOduration=1.583655574 podStartE2EDuration="1.583655574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:52:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:52:24.582380628 +0000 UTC m=+2099.545473917" watchObservedRunningTime="2026-07-24 03:52:24.583655574 +0000 UTC m=+2099.546748862"
Jul 24 03:52:25.606373 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:25.606347    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:52:25.611317 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:25.611296    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:52:34.789008 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:34.788964    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg"]
Jul 24 03:52:34.792974 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:34.792948    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg"
Jul 24 03:52:34.795421 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:34.795392    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 03:52:34.795567 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:34.795453    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 03:52:34.799903 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:34.799871    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg"]
Jul 24 03:52:34.863596 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:34.863556    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-365be7db-cmqgg\" (UID: \"a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg"
Jul 24 03:52:34.964574 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:34.964530    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-365be7db-cmqgg\" (UID: \"a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg"
Jul 24 03:52:34.965197 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:34.965175    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-365be7db-cmqgg\" (UID: \"a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg"
Jul 24 03:52:35.106530 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:35.106396    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg"
Jul 24 03:52:35.246034 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:35.246008    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg"]
Jul 24 03:52:35.248742 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:52:35.248708    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a33d9f_bf05_4cbf_a2cf_411cf06eff8b.slice/crio-76f113e2475716f98650d9d7e5e4854ff029d929669e2fde5b27a371cbd1b735 WatchSource:0}: Error finding container 76f113e2475716f98650d9d7e5e4854ff029d929669e2fde5b27a371cbd1b735: Status 404 returned error can't find the container with id 76f113e2475716f98650d9d7e5e4854ff029d929669e2fde5b27a371cbd1b735
Jul 24 03:52:35.560958 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:35.560927    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"
Jul 24 03:52:35.561340 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:35.561322    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"
Jul 24 03:52:35.612579 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:35.612532    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg" event={"ID":"a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b","Type":"ContainerStarted","Data":"e5417dd547c6dc84cbf37421341c8e867779867e49ca6e115a1485b3d3433853"}
Jul 24 03:52:35.612776 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:35.612611    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg" event={"ID":"a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b","Type":"ContainerStarted","Data":"76f113e2475716f98650d9d7e5e4854ff029d929669e2fde5b27a371cbd1b735"}
Jul 24 03:52:35.627169 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:35.627120    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg" podStartSLOduration=1.6270984469999998 podStartE2EDuration="1.627098447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:52:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:52:35.625102445 +0000 UTC m=+2110.588195732" watchObservedRunningTime="2026-07-24 03:52:35.627098447 +0000 UTC m=+2110.590191737"
Jul 24 03:52:36.619253 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:36.619218    2574 generic.go:356] "Generic (PLEG): container finished" podID="a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b" containerID="e5417dd547c6dc84cbf37421341c8e867779867e49ca6e115a1485b3d3433853" exitCode=0
Jul 24 03:52:36.619683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:36.619309    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg" event={"ID":"a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b","Type":"ContainerDied","Data":"e5417dd547c6dc84cbf37421341c8e867779867e49ca6e115a1485b3d3433853"}
Jul 24 03:52:37.761863 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:37.761836    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg"
Jul 24 03:52:37.896653 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:37.896622    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b-maas-api-service-ca\") pod \"a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b\" (UID: \"a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b\") "
Jul 24 03:52:37.897001 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:37.896976    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b" (UID: "a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:52:37.998106 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:37.998067    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b-maas-api-service-ca\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:52:38.631578 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:38.631544    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg"
Jul 24 03:52:38.631578 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:38.631554    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg" event={"ID":"a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b","Type":"ContainerDied","Data":"76f113e2475716f98650d9d7e5e4854ff029d929669e2fde5b27a371cbd1b735"}
Jul 24 03:52:38.631798 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:38.631606    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76f113e2475716f98650d9d7e5e4854ff029d929669e2fde5b27a371cbd1b735"
Jul 24 03:52:45.466862 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.466819    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"]
Jul 24 03:52:45.467350 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.467081    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2" podUID="ece4aa0b-66e5-4d9e-8992-64b472a0378c" containerName="payload-processing" containerID="cri-o://24bf4b72ef92c7ec391599e302a377359ff78c7f6eae5c5379ba062622132450" gracePeriod=30
Jul 24 03:52:45.472910 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.472882    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"]
Jul 24 03:52:45.473306 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.473276    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h" podUID="45373c2d-6207-4c1d-99ba-15cf810b3cb3" containerName="payload-pre-processing" containerID="cri-o://026c7fd7270d8fb6e596af1baecd18ab2cd5469f9d8af7fa5f8679b5a60b6d4c" gracePeriod=30
Jul 24 03:52:45.559091 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.559053    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2" podUID="ece4aa0b-66e5-4d9e-8992-64b472a0378c" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.49:9004: connect: connection refused"
Jul 24 03:52:45.559228 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.559053    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h" podUID="45373c2d-6207-4c1d-99ba-15cf810b3cb3" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.48:9004: connect: connection refused"
Jul 24 03:52:45.673653 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.673623    2574 generic.go:356] "Generic (PLEG): container finished" podID="45373c2d-6207-4c1d-99ba-15cf810b3cb3" containerID="026c7fd7270d8fb6e596af1baecd18ab2cd5469f9d8af7fa5f8679b5a60b6d4c" exitCode=0
Jul 24 03:52:45.673841 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.673698    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h" event={"ID":"45373c2d-6207-4c1d-99ba-15cf810b3cb3","Type":"ContainerDied","Data":"026c7fd7270d8fb6e596af1baecd18ab2cd5469f9d8af7fa5f8679b5a60b6d4c"}
Jul 24 03:52:45.675211 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.675185    2574 generic.go:356] "Generic (PLEG): container finished" podID="ece4aa0b-66e5-4d9e-8992-64b472a0378c" containerID="24bf4b72ef92c7ec391599e302a377359ff78c7f6eae5c5379ba062622132450" exitCode=0
Jul 24 03:52:45.675318 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.675221    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2" event={"ID":"ece4aa0b-66e5-4d9e-8992-64b472a0378c","Type":"ContainerDied","Data":"24bf4b72ef92c7ec391599e302a377359ff78c7f6eae5c5379ba062622132450"}
Jul 24 03:52:45.743450 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.743399    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"
Jul 24 03:52:45.754893 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.754870    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"
Jul 24 03:52:45.784146 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.784112    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjlmb\" (UniqueName: \"kubernetes.io/projected/ece4aa0b-66e5-4d9e-8992-64b472a0378c-kube-api-access-bjlmb\") pod \"ece4aa0b-66e5-4d9e-8992-64b472a0378c\" (UID: \"ece4aa0b-66e5-4d9e-8992-64b472a0378c\") "
Jul 24 03:52:45.784146 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.784151    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9ttx\" (UniqueName: \"kubernetes.io/projected/45373c2d-6207-4c1d-99ba-15cf810b3cb3-kube-api-access-v9ttx\") pod \"45373c2d-6207-4c1d-99ba-15cf810b3cb3\" (UID: \"45373c2d-6207-4c1d-99ba-15cf810b3cb3\") "
Jul 24 03:52:45.784344 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.784210    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ece4aa0b-66e5-4d9e-8992-64b472a0378c-plugins-config-volume\") pod \"ece4aa0b-66e5-4d9e-8992-64b472a0378c\" (UID: \"ece4aa0b-66e5-4d9e-8992-64b472a0378c\") "
Jul 24 03:52:45.784344 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.784271    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45373c2d-6207-4c1d-99ba-15cf810b3cb3-plugins-config-volume\") pod \"45373c2d-6207-4c1d-99ba-15cf810b3cb3\" (UID: \"45373c2d-6207-4c1d-99ba-15cf810b3cb3\") "
Jul 24 03:52:45.784683 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.784652    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ece4aa0b-66e5-4d9e-8992-64b472a0378c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ece4aa0b-66e5-4d9e-8992-64b472a0378c" (UID: "ece4aa0b-66e5-4d9e-8992-64b472a0378c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:52:45.784823 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.784783    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45373c2d-6207-4c1d-99ba-15cf810b3cb3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "45373c2d-6207-4c1d-99ba-15cf810b3cb3" (UID: "45373c2d-6207-4c1d-99ba-15cf810b3cb3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:52:45.786443 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.786394    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ece4aa0b-66e5-4d9e-8992-64b472a0378c-kube-api-access-bjlmb" (OuterVolumeSpecName: "kube-api-access-bjlmb") pod "ece4aa0b-66e5-4d9e-8992-64b472a0378c" (UID: "ece4aa0b-66e5-4d9e-8992-64b472a0378c"). InnerVolumeSpecName "kube-api-access-bjlmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:52:45.786564 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.786450    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45373c2d-6207-4c1d-99ba-15cf810b3cb3-kube-api-access-v9ttx" (OuterVolumeSpecName: "kube-api-access-v9ttx") pod "45373c2d-6207-4c1d-99ba-15cf810b3cb3" (UID: "45373c2d-6207-4c1d-99ba-15cf810b3cb3"). InnerVolumeSpecName "kube-api-access-v9ttx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:52:45.885383 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.885350    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ece4aa0b-66e5-4d9e-8992-64b472a0378c-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:52:45.885383 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.885378    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45373c2d-6207-4c1d-99ba-15cf810b3cb3-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:52:45.885383 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.885388    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjlmb\" (UniqueName: \"kubernetes.io/projected/ece4aa0b-66e5-4d9e-8992-64b472a0378c-kube-api-access-bjlmb\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:52:45.885647 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:45.885398    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9ttx\" (UniqueName: \"kubernetes.io/projected/45373c2d-6207-4c1d-99ba-15cf810b3cb3-kube-api-access-v9ttx\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:52:46.688502 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:46.688420    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"
Jul 24 03:52:46.688918 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:46.688420    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h" event={"ID":"45373c2d-6207-4c1d-99ba-15cf810b3cb3","Type":"ContainerDied","Data":"8d404fa6a7e5420052470ec88a91edca209cc412030fc2d520732544534f70bc"}
Jul 24 03:52:46.688918 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:46.688631    2574 scope.go:117] "RemoveContainer" containerID="026c7fd7270d8fb6e596af1baecd18ab2cd5469f9d8af7fa5f8679b5a60b6d4c"
Jul 24 03:52:46.690009 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:46.689891    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2" event={"ID":"ece4aa0b-66e5-4d9e-8992-64b472a0378c","Type":"ContainerDied","Data":"86da213fcb6c94ee9b01bdd747b4a9c163ee104e19bf12707ec1a31275b6b0d8"}
Jul 24 03:52:46.690009 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:46.689910    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"
Jul 24 03:52:46.699082 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:46.699057    2574 scope.go:117] "RemoveContainer" containerID="24bf4b72ef92c7ec391599e302a377359ff78c7f6eae5c5379ba062622132450"
Jul 24 03:52:46.713327 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:46.713240    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"]
Jul 24 03:52:46.715523 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:46.715500    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-365be7db-574fccdb4-hsw4h"]
Jul 24 03:52:46.725373 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:46.725344    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"]
Jul 24 03:52:46.727389 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:46.727367    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-365be7db-669b59bbdb-nqpx2"]
Jul 24 03:52:47.572825 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:47.572792    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45373c2d-6207-4c1d-99ba-15cf810b3cb3" path="/var/lib/kubelet/pods/45373c2d-6207-4c1d-99ba-15cf810b3cb3/volumes"
Jul 24 03:52:47.573242 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:47.573222    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ece4aa0b-66e5-4d9e-8992-64b472a0378c" path="/var/lib/kubelet/pods/ece4aa0b-66e5-4d9e-8992-64b472a0378c/volumes"
Jul 24 03:52:49.856253 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:49.856215    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg"]
Jul 24 03:52:49.858873 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:49.858846    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-365be7db-cmqgg"]
Jul 24 03:52:51.573003 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:52:51.572962    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b" path="/var/lib/kubelet/pods/a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b/volumes"
Jul 24 03:53:19.973887 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.973800    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"]
Jul 24 03:53:19.974725 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.974176    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece4aa0b-66e5-4d9e-8992-64b472a0378c" containerName="payload-processing"
Jul 24 03:53:19.974725 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.974190    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece4aa0b-66e5-4d9e-8992-64b472a0378c" containerName="payload-processing"
Jul 24 03:53:19.974725 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.974204    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45373c2d-6207-4c1d-99ba-15cf810b3cb3" containerName="payload-pre-processing"
Jul 24 03:53:19.974725 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.974213    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="45373c2d-6207-4c1d-99ba-15cf810b3cb3" containerName="payload-pre-processing"
Jul 24 03:53:19.974725 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.974227    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b" containerName="revoke-keys"
Jul 24 03:53:19.974725 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.974236    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b" containerName="revoke-keys"
Jul 24 03:53:19.974725 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.974329    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="45373c2d-6207-4c1d-99ba-15cf810b3cb3" containerName="payload-pre-processing"
Jul 24 03:53:19.974725 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.974341    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a33d9f-bf05-4cbf-a2cf-411cf06eff8b" containerName="revoke-keys"
Jul 24 03:53:19.974725 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.974351    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece4aa0b-66e5-4d9e-8992-64b472a0378c" containerName="payload-processing"
Jul 24 03:53:19.977937 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.977910    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"
Jul 24 03:53:19.980625 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.980602    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e2438b97\""
Jul 24 03:53:19.980745 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.980690    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e2438b97-dockercfg-5f85c\""
Jul 24 03:53:19.988243 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:19.988216    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"]
Jul 24 03:53:20.019014 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.018980    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"]
Jul 24 03:53:20.022612 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.022594    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"
Jul 24 03:53:20.042896 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.042853    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"]
Jul 24 03:53:20.113395 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.113355    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sgvr\" (UniqueName: \"kubernetes.io/projected/7a2f310a-30aa-49ce-883a-b93b4cda21f0-kube-api-access-4sgvr\") pod \"payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl\" (UID: \"7a2f310a-30aa-49ce-883a-b93b4cda21f0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"
Jul 24 03:53:20.113607 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.113470    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5vb9\" (UniqueName: \"kubernetes.io/projected/210db26e-0102-4c28-b324-3aa9a7b7fcd0-kube-api-access-z5vb9\") pod \"payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl\" (UID: \"210db26e-0102-4c28-b324-3aa9a7b7fcd0\") " pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"
Jul 24 03:53:20.113607 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.113499    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2f310a-30aa-49ce-883a-b93b4cda21f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl\" (UID: \"7a2f310a-30aa-49ce-883a-b93b4cda21f0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"
Jul 24 03:53:20.113607 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.113539    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/210db26e-0102-4c28-b324-3aa9a7b7fcd0-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl\" (UID: \"210db26e-0102-4c28-b324-3aa9a7b7fcd0\") " pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"
Jul 24 03:53:20.214509 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.214465    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2f310a-30aa-49ce-883a-b93b4cda21f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl\" (UID: \"7a2f310a-30aa-49ce-883a-b93b4cda21f0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"
Jul 24 03:53:20.214672 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.214525    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/210db26e-0102-4c28-b324-3aa9a7b7fcd0-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl\" (UID: \"210db26e-0102-4c28-b324-3aa9a7b7fcd0\") " pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"
Jul 24 03:53:20.214672 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.214608    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sgvr\" (UniqueName: \"kubernetes.io/projected/7a2f310a-30aa-49ce-883a-b93b4cda21f0-kube-api-access-4sgvr\") pod \"payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl\" (UID: \"7a2f310a-30aa-49ce-883a-b93b4cda21f0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"
Jul 24 03:53:20.214672 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.214649    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5vb9\" (UniqueName: \"kubernetes.io/projected/210db26e-0102-4c28-b324-3aa9a7b7fcd0-kube-api-access-z5vb9\") pod \"payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl\" (UID: \"210db26e-0102-4c28-b324-3aa9a7b7fcd0\") " pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"
Jul 24 03:53:20.215183 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.215164    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2f310a-30aa-49ce-883a-b93b4cda21f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl\" (UID: \"7a2f310a-30aa-49ce-883a-b93b4cda21f0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"
Jul 24 03:53:20.215303 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.215285    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/210db26e-0102-4c28-b324-3aa9a7b7fcd0-plugins-config-volume\") pod \"payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl\" (UID: \"210db26e-0102-4c28-b324-3aa9a7b7fcd0\") " pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"
Jul 24 03:53:20.222952 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.222920    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sgvr\" (UniqueName: \"kubernetes.io/projected/7a2f310a-30aa-49ce-883a-b93b4cda21f0-kube-api-access-4sgvr\") pod \"payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl\" (UID: \"7a2f310a-30aa-49ce-883a-b93b4cda21f0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"
Jul 24 03:53:20.223217 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.223194    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5vb9\" (UniqueName: \"kubernetes.io/projected/210db26e-0102-4c28-b324-3aa9a7b7fcd0-kube-api-access-z5vb9\") pod \"payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl\" (UID: \"210db26e-0102-4c28-b324-3aa9a7b7fcd0\") " pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"
Jul 24 03:53:20.288948 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.288854    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"
Jul 24 03:53:20.332416 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.332372    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"
Jul 24 03:53:20.451486 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.451452    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"]
Jul 24 03:53:20.453174 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:53:20.453129    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a2f310a_30aa_49ce_883a_b93b4cda21f0.slice/crio-512c7d74938060e0b9001537e91f366f019dfe833fe55c88ecdca71bc01ae5cf WatchSource:0}: Error finding container 512c7d74938060e0b9001537e91f366f019dfe833fe55c88ecdca71bc01ae5cf: Status 404 returned error can't find the container with id 512c7d74938060e0b9001537e91f366f019dfe833fe55c88ecdca71bc01ae5cf
Jul 24 03:53:20.496293 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.496269    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"]
Jul 24 03:53:20.500549 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:53:20.500524    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210db26e_0102_4c28_b324_3aa9a7b7fcd0.slice/crio-be782ecb27aa342f53d28766829ff6d2ed8c4bcf8d9d9bcccde89af6019e574e WatchSource:0}: Error finding container be782ecb27aa342f53d28766829ff6d2ed8c4bcf8d9d9bcccde89af6019e574e: Status 404 returned error can't find the container with id be782ecb27aa342f53d28766829ff6d2ed8c4bcf8d9d9bcccde89af6019e574e
Jul 24 03:53:20.864533 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.864412    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl" event={"ID":"210db26e-0102-4c28-b324-3aa9a7b7fcd0","Type":"ContainerStarted","Data":"847b90bf0e0bc74579407d49f94f6bb32f78f5831e4f7af90f70071fece974c1"}
Jul 24 03:53:20.864533 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.864486    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl" event={"ID":"210db26e-0102-4c28-b324-3aa9a7b7fcd0","Type":"ContainerStarted","Data":"be782ecb27aa342f53d28766829ff6d2ed8c4bcf8d9d9bcccde89af6019e574e"}
Jul 24 03:53:20.864759 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.864603    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"
Jul 24 03:53:20.866083 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.866060    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl" event={"ID":"7a2f310a-30aa-49ce-883a-b93b4cda21f0","Type":"ContainerStarted","Data":"a33540af58be397e85c41e2b09f2cbe33da44905facd637a6b8a9f23c512c294"}
Jul 24 03:53:20.866205 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.866098    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"
Jul 24 03:53:20.866205 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.866114    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl" event={"ID":"7a2f310a-30aa-49ce-883a-b93b4cda21f0","Type":"ContainerStarted","Data":"512c7d74938060e0b9001537e91f366f019dfe833fe55c88ecdca71bc01ae5cf"}
Jul 24 03:53:20.880723 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.880670    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl" podStartSLOduration=1.880656004 podStartE2EDuration="1.880656004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:53:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:53:20.879156046 +0000 UTC m=+2155.842249356" watchObservedRunningTime="2026-07-24 03:53:20.880656004 +0000 UTC m=+2155.843749290"
Jul 24 03:53:20.894932 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:20.894882    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl" podStartSLOduration=1.894866045 podStartE2EDuration="1.894866045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:53:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:53:20.893672064 +0000 UTC m=+2155.856765351" watchObservedRunningTime="2026-07-24 03:53:20.894866045 +0000 UTC m=+2155.857959407"
Jul 24 03:53:31.874140 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:31.874105    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"
Jul 24 03:53:31.874631 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:53:31.874199    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"
Jul 24 03:54:03.243294 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:03.243256    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx"]
Jul 24 03:54:03.246889 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:03.246868    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx"
Jul 24 03:54:03.249243 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:03.249216    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 03:54:03.249349 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:03.249216    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 03:54:03.254609 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:03.254588    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx"]
Jul 24 03:54:03.331322 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:03.331284    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f45b441-e88c-4cee-a095-1c39999b18d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e2438b97-jslhx\" (UID: \"2f45b441-e88c-4cee-a095-1c39999b18d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx"
Jul 24 03:54:03.431904 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:03.431871    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f45b441-e88c-4cee-a095-1c39999b18d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e2438b97-jslhx\" (UID: \"2f45b441-e88c-4cee-a095-1c39999b18d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx"
Jul 24 03:54:03.432529 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:03.432506    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f45b441-e88c-4cee-a095-1c39999b18d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e2438b97-jslhx\" (UID: \"2f45b441-e88c-4cee-a095-1c39999b18d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx"
Jul 24 03:54:03.558680 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:03.558590    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx"
Jul 24 03:54:03.697414 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:03.697377    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx"]
Jul 24 03:54:03.700971 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:54:03.700943    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f45b441_e88c_4cee_a095_1c39999b18d3.slice/crio-635b9cb898bee0e675eb25e275f186c782e607ab46eeae49e1b1b58e32337543 WatchSource:0}: Error finding container 635b9cb898bee0e675eb25e275f186c782e607ab46eeae49e1b1b58e32337543: Status 404 returned error can't find the container with id 635b9cb898bee0e675eb25e275f186c782e607ab46eeae49e1b1b58e32337543
Jul 24 03:54:04.100835 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:04.100799    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx" event={"ID":"2f45b441-e88c-4cee-a095-1c39999b18d3","Type":"ContainerStarted","Data":"8f493c0481c2c1e914e61541e5923b0d6bd1cf8e97ebd3a1653fd179567aa0d7"}
Jul 24 03:54:04.101027 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:04.100859    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx" event={"ID":"2f45b441-e88c-4cee-a095-1c39999b18d3","Type":"ContainerStarted","Data":"635b9cb898bee0e675eb25e275f186c782e607ab46eeae49e1b1b58e32337543"}
Jul 24 03:54:04.117552 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:04.117504    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx" podStartSLOduration=1.117486663 podStartE2EDuration="1.117486663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:54:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:54:04.116002104 +0000 UTC m=+2199.079095393" watchObservedRunningTime="2026-07-24 03:54:04.117486663 +0000 UTC m=+2199.080579983"
Jul 24 03:54:05.107187 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:05.107155    2574 generic.go:356] "Generic (PLEG): container finished" podID="2f45b441-e88c-4cee-a095-1c39999b18d3" containerID="8f493c0481c2c1e914e61541e5923b0d6bd1cf8e97ebd3a1653fd179567aa0d7" exitCode=0
Jul 24 03:54:05.107610 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:05.107249    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx" event={"ID":"2f45b441-e88c-4cee-a095-1c39999b18d3","Type":"ContainerDied","Data":"8f493c0481c2c1e914e61541e5923b0d6bd1cf8e97ebd3a1653fd179567aa0d7"}
Jul 24 03:54:06.251897 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:06.251870    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx"
Jul 24 03:54:06.255177 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:06.255155    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f45b441-e88c-4cee-a095-1c39999b18d3-maas-api-service-ca\") pod \"2f45b441-e88c-4cee-a095-1c39999b18d3\" (UID: \"2f45b441-e88c-4cee-a095-1c39999b18d3\") "
Jul 24 03:54:06.255485 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:06.255466    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f45b441-e88c-4cee-a095-1c39999b18d3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2f45b441-e88c-4cee-a095-1c39999b18d3" (UID: "2f45b441-e88c-4cee-a095-1c39999b18d3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:54:06.355906 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:06.355860    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f45b441-e88c-4cee-a095-1c39999b18d3-maas-api-service-ca\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:54:07.118829 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:07.118788    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx" event={"ID":"2f45b441-e88c-4cee-a095-1c39999b18d3","Type":"ContainerDied","Data":"635b9cb898bee0e675eb25e275f186c782e607ab46eeae49e1b1b58e32337543"}
Jul 24 03:54:07.119023 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:07.118838    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="635b9cb898bee0e675eb25e275f186c782e607ab46eeae49e1b1b58e32337543"
Jul 24 03:54:07.119023 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:07.118802    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx"
Jul 24 03:54:18.782837 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:18.782805    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"]
Jul 24 03:54:18.783301 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:18.783019    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl" podUID="210db26e-0102-4c28-b324-3aa9a7b7fcd0" containerName="payload-processing" containerID="cri-o://847b90bf0e0bc74579407d49f94f6bb32f78f5831e4f7af90f70071fece974c1" gracePeriod=30
Jul 24 03:54:18.790146 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:18.790115    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"]
Jul 24 03:54:18.790354 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:18.790333    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl" podUID="7a2f310a-30aa-49ce-883a-b93b4cda21f0" containerName="payload-pre-processing" containerID="cri-o://a33540af58be397e85c41e2b09f2cbe33da44905facd637a6b8a9f23c512c294" gracePeriod=30
Jul 24 03:54:19.088807 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.088779    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"
Jul 24 03:54:19.091602 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.091582    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"
Jul 24 03:54:19.179941 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.179910    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/210db26e-0102-4c28-b324-3aa9a7b7fcd0-plugins-config-volume\") pod \"210db26e-0102-4c28-b324-3aa9a7b7fcd0\" (UID: \"210db26e-0102-4c28-b324-3aa9a7b7fcd0\") "
Jul 24 03:54:19.180124 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.179953    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sgvr\" (UniqueName: \"kubernetes.io/projected/7a2f310a-30aa-49ce-883a-b93b4cda21f0-kube-api-access-4sgvr\") pod \"7a2f310a-30aa-49ce-883a-b93b4cda21f0\" (UID: \"7a2f310a-30aa-49ce-883a-b93b4cda21f0\") "
Jul 24 03:54:19.180124 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.179989    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5vb9\" (UniqueName: \"kubernetes.io/projected/210db26e-0102-4c28-b324-3aa9a7b7fcd0-kube-api-access-z5vb9\") pod \"210db26e-0102-4c28-b324-3aa9a7b7fcd0\" (UID: \"210db26e-0102-4c28-b324-3aa9a7b7fcd0\") "
Jul 24 03:54:19.180124 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.180011    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2f310a-30aa-49ce-883a-b93b4cda21f0-plugins-config-volume\") pod \"7a2f310a-30aa-49ce-883a-b93b4cda21f0\" (UID: \"7a2f310a-30aa-49ce-883a-b93b4cda21f0\") "
Jul 24 03:54:19.180393 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.180363    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/210db26e-0102-4c28-b324-3aa9a7b7fcd0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "210db26e-0102-4c28-b324-3aa9a7b7fcd0" (UID: "210db26e-0102-4c28-b324-3aa9a7b7fcd0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:54:19.180536 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.180515    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a2f310a-30aa-49ce-883a-b93b4cda21f0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a2f310a-30aa-49ce-883a-b93b4cda21f0" (UID: "7a2f310a-30aa-49ce-883a-b93b4cda21f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:54:19.182447 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.182412    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a2f310a-30aa-49ce-883a-b93b4cda21f0-kube-api-access-4sgvr" (OuterVolumeSpecName: "kube-api-access-4sgvr") pod "7a2f310a-30aa-49ce-883a-b93b4cda21f0" (UID: "7a2f310a-30aa-49ce-883a-b93b4cda21f0"). InnerVolumeSpecName "kube-api-access-4sgvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:54:19.182447 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.182418    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/210db26e-0102-4c28-b324-3aa9a7b7fcd0-kube-api-access-z5vb9" (OuterVolumeSpecName: "kube-api-access-z5vb9") pod "210db26e-0102-4c28-b324-3aa9a7b7fcd0" (UID: "210db26e-0102-4c28-b324-3aa9a7b7fcd0"). InnerVolumeSpecName "kube-api-access-z5vb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:54:19.189109 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.189084    2574 generic.go:356] "Generic (PLEG): container finished" podID="210db26e-0102-4c28-b324-3aa9a7b7fcd0" containerID="847b90bf0e0bc74579407d49f94f6bb32f78f5831e4f7af90f70071fece974c1" exitCode=0
Jul 24 03:54:19.189228 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.189129    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"
Jul 24 03:54:19.189228 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.189165    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl" event={"ID":"210db26e-0102-4c28-b324-3aa9a7b7fcd0","Type":"ContainerDied","Data":"847b90bf0e0bc74579407d49f94f6bb32f78f5831e4f7af90f70071fece974c1"}
Jul 24 03:54:19.189228 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.189221    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl" event={"ID":"210db26e-0102-4c28-b324-3aa9a7b7fcd0","Type":"ContainerDied","Data":"be782ecb27aa342f53d28766829ff6d2ed8c4bcf8d9d9bcccde89af6019e574e"}
Jul 24 03:54:19.189380 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.189253    2574 scope.go:117] "RemoveContainer" containerID="847b90bf0e0bc74579407d49f94f6bb32f78f5831e4f7af90f70071fece974c1"
Jul 24 03:54:19.190572 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.190547    2574 generic.go:356] "Generic (PLEG): container finished" podID="7a2f310a-30aa-49ce-883a-b93b4cda21f0" containerID="a33540af58be397e85c41e2b09f2cbe33da44905facd637a6b8a9f23c512c294" exitCode=0
Jul 24 03:54:19.190673 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.190582    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl" event={"ID":"7a2f310a-30aa-49ce-883a-b93b4cda21f0","Type":"ContainerDied","Data":"a33540af58be397e85c41e2b09f2cbe33da44905facd637a6b8a9f23c512c294"}
Jul 24 03:54:19.190673 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.190612    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"
Jul 24 03:54:19.190673 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.190622    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl" event={"ID":"7a2f310a-30aa-49ce-883a-b93b4cda21f0","Type":"ContainerDied","Data":"512c7d74938060e0b9001537e91f366f019dfe833fe55c88ecdca71bc01ae5cf"}
Jul 24 03:54:19.198468 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.198442    2574 scope.go:117] "RemoveContainer" containerID="847b90bf0e0bc74579407d49f94f6bb32f78f5831e4f7af90f70071fece974c1"
Jul 24 03:54:19.198767 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:54:19.198746    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"847b90bf0e0bc74579407d49f94f6bb32f78f5831e4f7af90f70071fece974c1\": container with ID starting with 847b90bf0e0bc74579407d49f94f6bb32f78f5831e4f7af90f70071fece974c1 not found: ID does not exist" containerID="847b90bf0e0bc74579407d49f94f6bb32f78f5831e4f7af90f70071fece974c1"
Jul 24 03:54:19.198857 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.198775    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"847b90bf0e0bc74579407d49f94f6bb32f78f5831e4f7af90f70071fece974c1"} err="failed to get container status \"847b90bf0e0bc74579407d49f94f6bb32f78f5831e4f7af90f70071fece974c1\": rpc error: code = NotFound desc = could not find container \"847b90bf0e0bc74579407d49f94f6bb32f78f5831e4f7af90f70071fece974c1\": container with ID starting with 847b90bf0e0bc74579407d49f94f6bb32f78f5831e4f7af90f70071fece974c1 not found: ID does not exist"
Jul 24 03:54:19.198857 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.198793    2574 scope.go:117] "RemoveContainer" containerID="a33540af58be397e85c41e2b09f2cbe33da44905facd637a6b8a9f23c512c294"
Jul 24 03:54:19.207503 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.207478    2574 scope.go:117] "RemoveContainer" containerID="a33540af58be397e85c41e2b09f2cbe33da44905facd637a6b8a9f23c512c294"
Jul 24 03:54:19.207781 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:54:19.207763    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a33540af58be397e85c41e2b09f2cbe33da44905facd637a6b8a9f23c512c294\": container with ID starting with a33540af58be397e85c41e2b09f2cbe33da44905facd637a6b8a9f23c512c294 not found: ID does not exist" containerID="a33540af58be397e85c41e2b09f2cbe33da44905facd637a6b8a9f23c512c294"
Jul 24 03:54:19.207837 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.207793    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a33540af58be397e85c41e2b09f2cbe33da44905facd637a6b8a9f23c512c294"} err="failed to get container status \"a33540af58be397e85c41e2b09f2cbe33da44905facd637a6b8a9f23c512c294\": rpc error: code = NotFound desc = could not find container \"a33540af58be397e85c41e2b09f2cbe33da44905facd637a6b8a9f23c512c294\": container with ID starting with a33540af58be397e85c41e2b09f2cbe33da44905facd637a6b8a9f23c512c294 not found: ID does not exist"
Jul 24 03:54:19.217926 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.217902    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"]
Jul 24 03:54:19.223263 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.223242    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e2438b97-58969bf8b-6bfdl"]
Jul 24 03:54:19.233655 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.233629    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"]
Jul 24 03:54:19.235390 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.235373    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e2438b97-b5d575c76-rnvfl"]
Jul 24 03:54:19.281295 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.281265    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/210db26e-0102-4c28-b324-3aa9a7b7fcd0-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:54:19.281295 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.281293    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sgvr\" (UniqueName: \"kubernetes.io/projected/7a2f310a-30aa-49ce-883a-b93b4cda21f0-kube-api-access-4sgvr\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:54:19.281528 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.281305    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5vb9\" (UniqueName: \"kubernetes.io/projected/210db26e-0102-4c28-b324-3aa9a7b7fcd0-kube-api-access-z5vb9\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:54:19.281528 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.281314    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2f310a-30aa-49ce-883a-b93b4cda21f0-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:54:19.572564 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.572531    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="210db26e-0102-4c28-b324-3aa9a7b7fcd0" path="/var/lib/kubelet/pods/210db26e-0102-4c28-b324-3aa9a7b7fcd0/volumes"
Jul 24 03:54:19.572897 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:19.572885    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a2f310a-30aa-49ce-883a-b93b4cda21f0" path="/var/lib/kubelet/pods/7a2f310a-30aa-49ce-883a-b93b4cda21f0/volumes"
Jul 24 03:54:23.317802 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:23.317756    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx"]
Jul 24 03:54:23.319708 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:23.319688    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e2438b97-jslhx"]
Jul 24 03:54:23.573262 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:23.573178    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f45b441-e88c-4cee-a095-1c39999b18d3" path="/var/lib/kubelet/pods/2f45b441-e88c-4cee-a095-1c39999b18d3/volumes"
Jul 24 03:54:25.696441 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:54:25.696391    2574 scope.go:117] "RemoveContainer" containerID="89841280c3aaf3e32154576a377b193252398ba228c6e50b9a903f1b43d1b0ff"
Jul 24 03:55:25.723738 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:55:25.723704    2574 scope.go:117] "RemoveContainer" containerID="d0d67010dd8f3c30572e2ac4033866c78c75e005e0fcb6b015302c2f146a56af"
Jul 24 03:56:21.179446 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.179346    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"]
Jul 24 03:56:21.181941 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.179761    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a2f310a-30aa-49ce-883a-b93b4cda21f0" containerName="payload-pre-processing"
Jul 24 03:56:21.181941 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.179777    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a2f310a-30aa-49ce-883a-b93b4cda21f0" containerName="payload-pre-processing"
Jul 24 03:56:21.181941 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.179795    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="210db26e-0102-4c28-b324-3aa9a7b7fcd0" containerName="payload-processing"
Jul 24 03:56:21.181941 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.179802    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="210db26e-0102-4c28-b324-3aa9a7b7fcd0" containerName="payload-processing"
Jul 24 03:56:21.181941 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.179810    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f45b441-e88c-4cee-a095-1c39999b18d3" containerName="revoke-keys"
Jul 24 03:56:21.181941 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.179816    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f45b441-e88c-4cee-a095-1c39999b18d3" containerName="revoke-keys"
Jul 24 03:56:21.181941 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.179885    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f45b441-e88c-4cee-a095-1c39999b18d3" containerName="revoke-keys"
Jul 24 03:56:21.181941 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.179897    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a2f310a-30aa-49ce-883a-b93b4cda21f0" containerName="payload-pre-processing"
Jul 24 03:56:21.181941 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.179902    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="210db26e-0102-4c28-b324-3aa9a7b7fcd0" containerName="payload-processing"
Jul 24 03:56:21.182837 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.182820    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"
Jul 24 03:56:21.185255 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.185234    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-93dff3-dockercfg-pspn2\""
Jul 24 03:56:21.185370 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.185234    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-93dff3\""
Jul 24 03:56:21.192126 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.192098    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"]
Jul 24 03:56:21.207413 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.207381    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"]
Jul 24 03:56:21.210930 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.210900    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"
Jul 24 03:56:21.222197 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.222167    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"]
Jul 24 03:56:21.336138 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.336087    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmvxz\" (UniqueName: \"kubernetes.io/projected/fd1971ca-d61b-402e-8fa1-bde9d53167af-kube-api-access-zmvxz\") pod \"payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q\" (UID: \"fd1971ca-d61b-402e-8fa1-bde9d53167af\") " pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"
Jul 24 03:56:21.336337 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.336165    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnxlk\" (UniqueName: \"kubernetes.io/projected/a5daf161-9aae-4597-9610-243d7689ae30-kube-api-access-bnxlk\") pod \"payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96\" (UID: \"a5daf161-9aae-4597-9610-243d7689ae30\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"
Jul 24 03:56:21.336337 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.336242    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5daf161-9aae-4597-9610-243d7689ae30-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96\" (UID: \"a5daf161-9aae-4597-9610-243d7689ae30\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"
Jul 24 03:56:21.336472 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.336360    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd1971ca-d61b-402e-8fa1-bde9d53167af-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q\" (UID: \"fd1971ca-d61b-402e-8fa1-bde9d53167af\") " pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"
Jul 24 03:56:21.437410 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.437316    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnxlk\" (UniqueName: \"kubernetes.io/projected/a5daf161-9aae-4597-9610-243d7689ae30-kube-api-access-bnxlk\") pod \"payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96\" (UID: \"a5daf161-9aae-4597-9610-243d7689ae30\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"
Jul 24 03:56:21.437410 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.437371    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5daf161-9aae-4597-9610-243d7689ae30-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96\" (UID: \"a5daf161-9aae-4597-9610-243d7689ae30\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"
Jul 24 03:56:21.437674 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.437500    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd1971ca-d61b-402e-8fa1-bde9d53167af-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q\" (UID: \"fd1971ca-d61b-402e-8fa1-bde9d53167af\") " pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"
Jul 24 03:56:21.437674 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.437575    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmvxz\" (UniqueName: \"kubernetes.io/projected/fd1971ca-d61b-402e-8fa1-bde9d53167af-kube-api-access-zmvxz\") pod \"payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q\" (UID: \"fd1971ca-d61b-402e-8fa1-bde9d53167af\") " pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"
Jul 24 03:56:21.438231 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.438208    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5daf161-9aae-4597-9610-243d7689ae30-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96\" (UID: \"a5daf161-9aae-4597-9610-243d7689ae30\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"
Jul 24 03:56:21.438332 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.438251    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd1971ca-d61b-402e-8fa1-bde9d53167af-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q\" (UID: \"fd1971ca-d61b-402e-8fa1-bde9d53167af\") " pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"
Jul 24 03:56:21.447320 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.447297    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmvxz\" (UniqueName: \"kubernetes.io/projected/fd1971ca-d61b-402e-8fa1-bde9d53167af-kube-api-access-zmvxz\") pod \"payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q\" (UID: \"fd1971ca-d61b-402e-8fa1-bde9d53167af\") " pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"
Jul 24 03:56:21.447455 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.447401    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnxlk\" (UniqueName: \"kubernetes.io/projected/a5daf161-9aae-4597-9610-243d7689ae30-kube-api-access-bnxlk\") pod \"payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96\" (UID: \"a5daf161-9aae-4597-9610-243d7689ae30\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"
Jul 24 03:56:21.495130 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.495080    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"
Jul 24 03:56:21.524109 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.524077    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"
Jul 24 03:56:21.659360 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.659323    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"]
Jul 24 03:56:21.660571 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:56:21.660541    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5daf161_9aae_4597_9610_243d7689ae30.slice/crio-e6a0e3649beb8b94b8f8a9e9c37cbacfe6c9eb4b6d54528670761513c38aaf00 WatchSource:0}: Error finding container e6a0e3649beb8b94b8f8a9e9c37cbacfe6c9eb4b6d54528670761513c38aaf00: Status 404 returned error can't find the container with id e6a0e3649beb8b94b8f8a9e9c37cbacfe6c9eb4b6d54528670761513c38aaf00
Jul 24 03:56:21.686909 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.686884    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"]
Jul 24 03:56:21.690665 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:56:21.690638    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1971ca_d61b_402e_8fa1_bde9d53167af.slice/crio-bc2129167ffd3105fa91fc3306877622069b95ed5b1367e199472c534056ff5d WatchSource:0}: Error finding container bc2129167ffd3105fa91fc3306877622069b95ed5b1367e199472c534056ff5d: Status 404 returned error can't find the container with id bc2129167ffd3105fa91fc3306877622069b95ed5b1367e199472c534056ff5d
Jul 24 03:56:21.818164 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.818123    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96" event={"ID":"a5daf161-9aae-4597-9610-243d7689ae30","Type":"ContainerStarted","Data":"55d7c99d41e8d50eede0d10b679d58744fe096954c639be4b8dcb4bbfb6f42d2"}
Jul 24 03:56:21.818328 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.818201    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"
Jul 24 03:56:21.818328 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.818229    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96" event={"ID":"a5daf161-9aae-4597-9610-243d7689ae30","Type":"ContainerStarted","Data":"e6a0e3649beb8b94b8f8a9e9c37cbacfe6c9eb4b6d54528670761513c38aaf00"}
Jul 24 03:56:21.819664 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.819637    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q" event={"ID":"fd1971ca-d61b-402e-8fa1-bde9d53167af","Type":"ContainerStarted","Data":"da5336ba79f10cc58a3a4085800a2fd3a6716d5981f4c9a7508d62cf148206cc"}
Jul 24 03:56:21.819771 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.819687    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q" event={"ID":"fd1971ca-d61b-402e-8fa1-bde9d53167af","Type":"ContainerStarted","Data":"bc2129167ffd3105fa91fc3306877622069b95ed5b1367e199472c534056ff5d"}
Jul 24 03:56:21.819771 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.819746    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"
Jul 24 03:56:21.840394 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.840343    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96" podStartSLOduration=0.840325636 podStartE2EDuration="840.325636ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:56:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:56:21.837128184 +0000 UTC m=+2336.800221471" watchObservedRunningTime="2026-07-24 03:56:21.840325636 +0000 UTC m=+2336.803418930"
Jul 24 03:56:21.853028 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:21.852975    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q" podStartSLOduration=0.852953144 podStartE2EDuration="852.953144ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:56:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:56:21.852211873 +0000 UTC m=+2336.815305161" watchObservedRunningTime="2026-07-24 03:56:21.852953144 +0000 UTC m=+2336.816046431"
Jul 24 03:56:25.745041 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:25.745004    2574 scope.go:117] "RemoveContainer" containerID="c473874911f677e27a73416a764e99d1f13974dd0fd64e0d4734c1d8bb49a2a7"
Jul 24 03:56:29.079975 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.079938    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"]
Jul 24 03:56:29.084970 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.084946    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"
Jul 24 03:56:29.087701 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.087678    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-b0d347\""
Jul 24 03:56:29.088066 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.088044    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-b0d347-dockercfg-kb4w6\""
Jul 24 03:56:29.101458 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.101413    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"]
Jul 24 03:56:29.167624 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.167585    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"]
Jul 24 03:56:29.171190 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.171168    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"
Jul 24 03:56:29.192337 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.192310    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"]
Jul 24 03:56:29.216814 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.216778    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d4348ef-f155-49ea-90b4-542569afa11f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7\" (UID: \"1d4348ef-f155-49ea-90b4-542569afa11f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"
Jul 24 03:56:29.216993 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.216818    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsmzt\" (UniqueName: \"kubernetes.io/projected/1d4348ef-f155-49ea-90b4-542569afa11f-kube-api-access-nsmzt\") pod \"payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7\" (UID: \"1d4348ef-f155-49ea-90b4-542569afa11f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"
Jul 24 03:56:29.317648 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.317606    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d4348ef-f155-49ea-90b4-542569afa11f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7\" (UID: \"1d4348ef-f155-49ea-90b4-542569afa11f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"
Jul 24 03:56:29.317648 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.317648    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsmzt\" (UniqueName: \"kubernetes.io/projected/1d4348ef-f155-49ea-90b4-542569afa11f-kube-api-access-nsmzt\") pod \"payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7\" (UID: \"1d4348ef-f155-49ea-90b4-542569afa11f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"
Jul 24 03:56:29.317898 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.317680    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qnrq\" (UniqueName: \"kubernetes.io/projected/ee570b27-a617-43d8-bfdb-787ad21a020a-kube-api-access-9qnrq\") pod \"payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn\" (UID: \"ee570b27-a617-43d8-bfdb-787ad21a020a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"
Jul 24 03:56:29.317898 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.317796    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee570b27-a617-43d8-bfdb-787ad21a020a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn\" (UID: \"ee570b27-a617-43d8-bfdb-787ad21a020a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"
Jul 24 03:56:29.318328 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.318286    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d4348ef-f155-49ea-90b4-542569afa11f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7\" (UID: \"1d4348ef-f155-49ea-90b4-542569afa11f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"
Jul 24 03:56:29.326126 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.326097    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsmzt\" (UniqueName: \"kubernetes.io/projected/1d4348ef-f155-49ea-90b4-542569afa11f-kube-api-access-nsmzt\") pod \"payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7\" (UID: \"1d4348ef-f155-49ea-90b4-542569afa11f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"
Jul 24 03:56:29.395039 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.395007    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"
Jul 24 03:56:29.418994 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.418961    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qnrq\" (UniqueName: \"kubernetes.io/projected/ee570b27-a617-43d8-bfdb-787ad21a020a-kube-api-access-9qnrq\") pod \"payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn\" (UID: \"ee570b27-a617-43d8-bfdb-787ad21a020a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"
Jul 24 03:56:29.419157 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.419044    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee570b27-a617-43d8-bfdb-787ad21a020a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn\" (UID: \"ee570b27-a617-43d8-bfdb-787ad21a020a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"
Jul 24 03:56:29.419753 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.419728    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee570b27-a617-43d8-bfdb-787ad21a020a-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn\" (UID: \"ee570b27-a617-43d8-bfdb-787ad21a020a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"
Jul 24 03:56:29.429644 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.429615    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qnrq\" (UniqueName: \"kubernetes.io/projected/ee570b27-a617-43d8-bfdb-787ad21a020a-kube-api-access-9qnrq\") pod \"payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn\" (UID: \"ee570b27-a617-43d8-bfdb-787ad21a020a\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"
Jul 24 03:56:29.482393 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.482366    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"
Jul 24 03:56:29.553804 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.553767    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"]
Jul 24 03:56:29.559323 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:56:29.558970    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4348ef_f155_49ea_90b4_542569afa11f.slice/crio-f977d7b03ae5c15d66011e7b91a6d5dbf35e6779644dadbadaf1cd2c7711d2e7 WatchSource:0}: Error finding container f977d7b03ae5c15d66011e7b91a6d5dbf35e6779644dadbadaf1cd2c7711d2e7: Status 404 returned error can't find the container with id f977d7b03ae5c15d66011e7b91a6d5dbf35e6779644dadbadaf1cd2c7711d2e7
Jul 24 03:56:29.646537 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.646512    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"]
Jul 24 03:56:29.649157 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:56:29.649129    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee570b27_a617_43d8_bfdb_787ad21a020a.slice/crio-7ff83f2e96baa0448edb14daae65568cf86edd2d56b3437a4fd3c7f34a100bbb WatchSource:0}: Error finding container 7ff83f2e96baa0448edb14daae65568cf86edd2d56b3437a4fd3c7f34a100bbb: Status 404 returned error can't find the container with id 7ff83f2e96baa0448edb14daae65568cf86edd2d56b3437a4fd3c7f34a100bbb
Jul 24 03:56:29.866338 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.866299    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn" event={"ID":"ee570b27-a617-43d8-bfdb-787ad21a020a","Type":"ContainerStarted","Data":"2ef772c79cfab73e52e499650f33e34a4c9fd88336178c4721b18381c6d39edf"}
Jul 24 03:56:29.866338 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.866368    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn" event={"ID":"ee570b27-a617-43d8-bfdb-787ad21a020a","Type":"ContainerStarted","Data":"7ff83f2e96baa0448edb14daae65568cf86edd2d56b3437a4fd3c7f34a100bbb"}
Jul 24 03:56:29.866719 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.866417    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"
Jul 24 03:56:29.868142 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.868119    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7" event={"ID":"1d4348ef-f155-49ea-90b4-542569afa11f","Type":"ContainerStarted","Data":"5df0998967340ed5220ccb9055ca87c745df77b7a7e660cfb8d0901d240a5e1e"}
Jul 24 03:56:29.868266 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.868163    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7" event={"ID":"1d4348ef-f155-49ea-90b4-542569afa11f","Type":"ContainerStarted","Data":"f977d7b03ae5c15d66011e7b91a6d5dbf35e6779644dadbadaf1cd2c7711d2e7"}
Jul 24 03:56:29.868266 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.868191    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"
Jul 24 03:56:29.882385 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.882331    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn" podStartSLOduration=0.88231467 podStartE2EDuration="882.31467ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:56:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:56:29.880726825 +0000 UTC m=+2344.843820115" watchObservedRunningTime="2026-07-24 03:56:29.88231467 +0000 UTC m=+2344.845407983"
Jul 24 03:56:29.895880 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:29.895826    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7" podStartSLOduration=0.895807247 podStartE2EDuration="895.807247ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:56:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:56:29.894918262 +0000 UTC m=+2344.858011546" watchObservedRunningTime="2026-07-24 03:56:29.895807247 +0000 UTC m=+2344.858900534"
Jul 24 03:56:32.827750 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:32.827716    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"
Jul 24 03:56:32.828194 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:32.827786    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"
Jul 24 03:56:40.874933 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:40.874899    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"
Jul 24 03:56:40.875346 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:56:40.874974    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"
Jul 24 03:57:25.638494 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:57:25.638459    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:57:25.647465 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:57:25.647440    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 03:57:25.766883 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:57:25.766847    2574 scope.go:117] "RemoveContainer" containerID="7eb050b2907ac2dc88cec0ee1ac845af07a0f23b89adac0abedf0042efd4f479"
Jul 24 03:58:25.069584 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:25.069542    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7"]
Jul 24 03:58:25.073191 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:25.073167    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7"
Jul 24 03:58:25.075764 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:25.075738    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 03:58:25.075884 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:25.075739    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 03:58:25.081734 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:25.081704    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7"]
Jul 24 03:58:25.234656 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:25.234618    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/739a7493-015a-4e3a-9042-956df55b4241-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7\" (UID: \"739a7493-015a-4e3a-9042-956df55b4241\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7"
Jul 24 03:58:25.335307 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:25.335196    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/739a7493-015a-4e3a-9042-956df55b4241-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7\" (UID: \"739a7493-015a-4e3a-9042-956df55b4241\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7"
Jul 24 03:58:25.336030 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:25.336003    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/739a7493-015a-4e3a-9042-956df55b4241-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7\" (UID: \"739a7493-015a-4e3a-9042-956df55b4241\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7"
Jul 24 03:58:25.387744 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:25.387708    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 03:58:25.396399 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:25.396366    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7"
Jul 24 03:58:25.555074 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:25.554900    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7"]
Jul 24 03:58:25.557857 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:58:25.557820    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod739a7493_015a_4e3a_9042_956df55b4241.slice/crio-bbc0469766218ff0e2312e91eb5913f29405f01e087119c5bba6450d65444816 WatchSource:0}: Error finding container bbc0469766218ff0e2312e91eb5913f29405f01e087119c5bba6450d65444816: Status 404 returned error can't find the container with id bbc0469766218ff0e2312e91eb5913f29405f01e087119c5bba6450d65444816
Jul 24 03:58:26.467245 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:26.467213    2574 generic.go:356] "Generic (PLEG): container finished" podID="739a7493-015a-4e3a-9042-956df55b4241" containerID="fd45f25e07183fb1f20a7850e5e05cb8ecc2a8793df05c9801c59e6d5c5a5b8e" exitCode=0
Jul 24 03:58:26.467694 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:26.467253    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7" event={"ID":"739a7493-015a-4e3a-9042-956df55b4241","Type":"ContainerDied","Data":"fd45f25e07183fb1f20a7850e5e05cb8ecc2a8793df05c9801c59e6d5c5a5b8e"}
Jul 24 03:58:26.467694 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:26.467295    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7" event={"ID":"739a7493-015a-4e3a-9042-956df55b4241","Type":"ContainerStarted","Data":"bbc0469766218ff0e2312e91eb5913f29405f01e087119c5bba6450d65444816"}
Jul 24 03:58:27.618137 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:27.618108    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7"
Jul 24 03:58:27.759839 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:27.759748    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/739a7493-015a-4e3a-9042-956df55b4241-maas-api-service-ca\") pod \"739a7493-015a-4e3a-9042-956df55b4241\" (UID: \"739a7493-015a-4e3a-9042-956df55b4241\") "
Jul 24 03:58:27.760101 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:27.760077    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/739a7493-015a-4e3a-9042-956df55b4241-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "739a7493-015a-4e3a-9042-956df55b4241" (UID: "739a7493-015a-4e3a-9042-956df55b4241"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:58:27.861417 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:27.861379    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/739a7493-015a-4e3a-9042-956df55b4241-maas-api-service-ca\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:58:28.478690 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:28.478656    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7"
Jul 24 03:58:28.478875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:28.478688    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7" event={"ID":"739a7493-015a-4e3a-9042-956df55b4241","Type":"ContainerDied","Data":"bbc0469766218ff0e2312e91eb5913f29405f01e087119c5bba6450d65444816"}
Jul 24 03:58:28.478875 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:28.478736    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbc0469766218ff0e2312e91eb5913f29405f01e087119c5bba6450d65444816"
Jul 24 03:58:35.627633 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:35.627597    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"]
Jul 24 03:58:35.628210 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:35.627851    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q" podUID="fd1971ca-d61b-402e-8fa1-bde9d53167af" containerName="payload-processing" containerID="cri-o://da5336ba79f10cc58a3a4085800a2fd3a6716d5981f4c9a7508d62cf148206cc" gracePeriod=30
Jul 24 03:58:35.634687 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:35.634077    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"]
Jul 24 03:58:35.634687 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:35.634302    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96" podUID="a5daf161-9aae-4597-9610-243d7689ae30" containerName="payload-pre-processing" containerID="cri-o://55d7c99d41e8d50eede0d10b679d58744fe096954c639be4b8dcb4bbfb6f42d2" gracePeriod=30
Jul 24 03:58:35.926797 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:35.926771    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"
Jul 24 03:58:35.930015 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:35.929992    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"
Jul 24 03:58:36.028332 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.028286    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmvxz\" (UniqueName: \"kubernetes.io/projected/fd1971ca-d61b-402e-8fa1-bde9d53167af-kube-api-access-zmvxz\") pod \"fd1971ca-d61b-402e-8fa1-bde9d53167af\" (UID: \"fd1971ca-d61b-402e-8fa1-bde9d53167af\") "
Jul 24 03:58:36.028332 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.028346    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd1971ca-d61b-402e-8fa1-bde9d53167af-plugins-config-volume\") pod \"fd1971ca-d61b-402e-8fa1-bde9d53167af\" (UID: \"fd1971ca-d61b-402e-8fa1-bde9d53167af\") "
Jul 24 03:58:36.028635 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.028371    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnxlk\" (UniqueName: \"kubernetes.io/projected/a5daf161-9aae-4597-9610-243d7689ae30-kube-api-access-bnxlk\") pod \"a5daf161-9aae-4597-9610-243d7689ae30\" (UID: \"a5daf161-9aae-4597-9610-243d7689ae30\") "
Jul 24 03:58:36.028635 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.028511    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5daf161-9aae-4597-9610-243d7689ae30-plugins-config-volume\") pod \"a5daf161-9aae-4597-9610-243d7689ae30\" (UID: \"a5daf161-9aae-4597-9610-243d7689ae30\") "
Jul 24 03:58:36.028782 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.028754    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd1971ca-d61b-402e-8fa1-bde9d53167af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd1971ca-d61b-402e-8fa1-bde9d53167af" (UID: "fd1971ca-d61b-402e-8fa1-bde9d53167af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:58:36.028913 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.028888    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5daf161-9aae-4597-9610-243d7689ae30-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a5daf161-9aae-4597-9610-243d7689ae30" (UID: "a5daf161-9aae-4597-9610-243d7689ae30"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:58:36.030667 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.030638    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd1971ca-d61b-402e-8fa1-bde9d53167af-kube-api-access-zmvxz" (OuterVolumeSpecName: "kube-api-access-zmvxz") pod "fd1971ca-d61b-402e-8fa1-bde9d53167af" (UID: "fd1971ca-d61b-402e-8fa1-bde9d53167af"). InnerVolumeSpecName "kube-api-access-zmvxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:58:36.030769 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.030747    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5daf161-9aae-4597-9610-243d7689ae30-kube-api-access-bnxlk" (OuterVolumeSpecName: "kube-api-access-bnxlk") pod "a5daf161-9aae-4597-9610-243d7689ae30" (UID: "a5daf161-9aae-4597-9610-243d7689ae30"). InnerVolumeSpecName "kube-api-access-bnxlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:58:36.129265 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.129238    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5daf161-9aae-4597-9610-243d7689ae30-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:58:36.129265 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.129265    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmvxz\" (UniqueName: \"kubernetes.io/projected/fd1971ca-d61b-402e-8fa1-bde9d53167af-kube-api-access-zmvxz\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:58:36.129265 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.129277    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd1971ca-d61b-402e-8fa1-bde9d53167af-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:58:36.129558 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.129292    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnxlk\" (UniqueName: \"kubernetes.io/projected/a5daf161-9aae-4597-9610-243d7689ae30-kube-api-access-bnxlk\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:58:36.520051 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.520003    2574 generic.go:356] "Generic (PLEG): container finished" podID="a5daf161-9aae-4597-9610-243d7689ae30" containerID="55d7c99d41e8d50eede0d10b679d58744fe096954c639be4b8dcb4bbfb6f42d2" exitCode=0
Jul 24 03:58:36.520238 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.520105    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"
Jul 24 03:58:36.520238 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.520145    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96" event={"ID":"a5daf161-9aae-4597-9610-243d7689ae30","Type":"ContainerDied","Data":"55d7c99d41e8d50eede0d10b679d58744fe096954c639be4b8dcb4bbfb6f42d2"}
Jul 24 03:58:36.520238 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.520204    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96" event={"ID":"a5daf161-9aae-4597-9610-243d7689ae30","Type":"ContainerDied","Data":"e6a0e3649beb8b94b8f8a9e9c37cbacfe6c9eb4b6d54528670761513c38aaf00"}
Jul 24 03:58:36.520238 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.520236    2574 scope.go:117] "RemoveContainer" containerID="55d7c99d41e8d50eede0d10b679d58744fe096954c639be4b8dcb4bbfb6f42d2"
Jul 24 03:58:36.521697 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.521674    2574 generic.go:356] "Generic (PLEG): container finished" podID="fd1971ca-d61b-402e-8fa1-bde9d53167af" containerID="da5336ba79f10cc58a3a4085800a2fd3a6716d5981f4c9a7508d62cf148206cc" exitCode=0
Jul 24 03:58:36.521819 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.521721    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q" event={"ID":"fd1971ca-d61b-402e-8fa1-bde9d53167af","Type":"ContainerDied","Data":"da5336ba79f10cc58a3a4085800a2fd3a6716d5981f4c9a7508d62cf148206cc"}
Jul 24 03:58:36.521819 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.521728    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"
Jul 24 03:58:36.521935 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.521767    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q" event={"ID":"fd1971ca-d61b-402e-8fa1-bde9d53167af","Type":"ContainerDied","Data":"bc2129167ffd3105fa91fc3306877622069b95ed5b1367e199472c534056ff5d"}
Jul 24 03:58:36.531238 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.531217    2574 scope.go:117] "RemoveContainer" containerID="55d7c99d41e8d50eede0d10b679d58744fe096954c639be4b8dcb4bbfb6f42d2"
Jul 24 03:58:36.531581 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:58:36.531547    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55d7c99d41e8d50eede0d10b679d58744fe096954c639be4b8dcb4bbfb6f42d2\": container with ID starting with 55d7c99d41e8d50eede0d10b679d58744fe096954c639be4b8dcb4bbfb6f42d2 not found: ID does not exist" containerID="55d7c99d41e8d50eede0d10b679d58744fe096954c639be4b8dcb4bbfb6f42d2"
Jul 24 03:58:36.531684 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.531583    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55d7c99d41e8d50eede0d10b679d58744fe096954c639be4b8dcb4bbfb6f42d2"} err="failed to get container status \"55d7c99d41e8d50eede0d10b679d58744fe096954c639be4b8dcb4bbfb6f42d2\": rpc error: code = NotFound desc = could not find container \"55d7c99d41e8d50eede0d10b679d58744fe096954c639be4b8dcb4bbfb6f42d2\": container with ID starting with 55d7c99d41e8d50eede0d10b679d58744fe096954c639be4b8dcb4bbfb6f42d2 not found: ID does not exist"
Jul 24 03:58:36.531684 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.531606    2574 scope.go:117] "RemoveContainer" containerID="da5336ba79f10cc58a3a4085800a2fd3a6716d5981f4c9a7508d62cf148206cc"
Jul 24 03:58:36.541012 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.540997    2574 scope.go:117] "RemoveContainer" containerID="da5336ba79f10cc58a3a4085800a2fd3a6716d5981f4c9a7508d62cf148206cc"
Jul 24 03:58:36.541295 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:58:36.541275    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da5336ba79f10cc58a3a4085800a2fd3a6716d5981f4c9a7508d62cf148206cc\": container with ID starting with da5336ba79f10cc58a3a4085800a2fd3a6716d5981f4c9a7508d62cf148206cc not found: ID does not exist" containerID="da5336ba79f10cc58a3a4085800a2fd3a6716d5981f4c9a7508d62cf148206cc"
Jul 24 03:58:36.541353 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.541304    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da5336ba79f10cc58a3a4085800a2fd3a6716d5981f4c9a7508d62cf148206cc"} err="failed to get container status \"da5336ba79f10cc58a3a4085800a2fd3a6716d5981f4c9a7508d62cf148206cc\": rpc error: code = NotFound desc = could not find container \"da5336ba79f10cc58a3a4085800a2fd3a6716d5981f4c9a7508d62cf148206cc\": container with ID starting with da5336ba79f10cc58a3a4085800a2fd3a6716d5981f4c9a7508d62cf148206cc not found: ID does not exist"
Jul 24 03:58:36.547056 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.547029    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"]
Jul 24 03:58:36.549838 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.549816    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-93dff3-667ffdb944-qhg96"]
Jul 24 03:58:36.560512 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.560485    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"]
Jul 24 03:58:36.564703 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:36.564673    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-93dff3-56dc779484-brc5q"]
Jul 24 03:58:37.573381 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:37.573348    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5daf161-9aae-4597-9610-243d7689ae30" path="/var/lib/kubelet/pods/a5daf161-9aae-4597-9610-243d7689ae30/volumes"
Jul 24 03:58:37.573794 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:37.573737    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd1971ca-d61b-402e-8fa1-bde9d53167af" path="/var/lib/kubelet/pods/fd1971ca-d61b-402e-8fa1-bde9d53167af/volumes"
Jul 24 03:58:40.158119 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:40.158083    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7"]
Jul 24 03:58:40.161367 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:40.161340    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-93dff3-zgfx7"]
Jul 24 03:58:41.572845 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:41.572802    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="739a7493-015a-4e3a-9042-956df55b4241" path="/var/lib/kubelet/pods/739a7493-015a-4e3a-9042-956df55b4241/volumes"
Jul 24 03:58:47.030077 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.030035    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r"]
Jul 24 03:58:47.030503 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.030489    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5daf161-9aae-4597-9610-243d7689ae30" containerName="payload-pre-processing"
Jul 24 03:58:47.030503 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.030501    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5daf161-9aae-4597-9610-243d7689ae30" containerName="payload-pre-processing"
Jul 24 03:58:47.030587 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.030522    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="739a7493-015a-4e3a-9042-956df55b4241" containerName="revoke-keys"
Jul 24 03:58:47.030587 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.030532    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="739a7493-015a-4e3a-9042-956df55b4241" containerName="revoke-keys"
Jul 24 03:58:47.030587 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.030544    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd1971ca-d61b-402e-8fa1-bde9d53167af" containerName="payload-processing"
Jul 24 03:58:47.030587 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.030552    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd1971ca-d61b-402e-8fa1-bde9d53167af" containerName="payload-processing"
Jul 24 03:58:47.030714 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.030626    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd1971ca-d61b-402e-8fa1-bde9d53167af" containerName="payload-processing"
Jul 24 03:58:47.030714 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.030637    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="739a7493-015a-4e3a-9042-956df55b4241" containerName="revoke-keys"
Jul 24 03:58:47.030714 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.030642    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5daf161-9aae-4597-9610-243d7689ae30" containerName="payload-pre-processing"
Jul 24 03:58:47.035159 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.035133    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r"
Jul 24 03:58:47.037454 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.037415    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 03:58:47.038004 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.037964    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 03:58:47.042402 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.042373    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r"]
Jul 24 03:58:47.138304 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.138257    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/063c0851-b09e-4695-85d5-c7f0543c8b6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r\" (UID: \"063c0851-b09e-4695-85d5-c7f0543c8b6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r"
Jul 24 03:58:47.239861 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.239821    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/063c0851-b09e-4695-85d5-c7f0543c8b6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r\" (UID: \"063c0851-b09e-4695-85d5-c7f0543c8b6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r"
Jul 24 03:58:47.240502 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.240480    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/063c0851-b09e-4695-85d5-c7f0543c8b6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r\" (UID: \"063c0851-b09e-4695-85d5-c7f0543c8b6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r"
Jul 24 03:58:47.349344 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.349256    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r"
Jul 24 03:58:47.513450 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.513394    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r"]
Jul 24 03:58:47.519566 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:58:47.519536    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod063c0851_b09e_4695_85d5_c7f0543c8b6f.slice/crio-b356fdedd7d7adbf7b25d26cfb78b7542d53245e80a37d7ebfdd5bc470f0999b WatchSource:0}: Error finding container b356fdedd7d7adbf7b25d26cfb78b7542d53245e80a37d7ebfdd5bc470f0999b: Status 404 returned error can't find the container with id b356fdedd7d7adbf7b25d26cfb78b7542d53245e80a37d7ebfdd5bc470f0999b
Jul 24 03:58:47.582024 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:47.581987    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r" event={"ID":"063c0851-b09e-4695-85d5-c7f0543c8b6f","Type":"ContainerStarted","Data":"b356fdedd7d7adbf7b25d26cfb78b7542d53245e80a37d7ebfdd5bc470f0999b"}
Jul 24 03:58:48.588696 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:48.588595    2574 generic.go:356] "Generic (PLEG): container finished" podID="063c0851-b09e-4695-85d5-c7f0543c8b6f" containerID="9c10af393728dbfe0937c964026bf55c939660ba0df15fe04347d8a8133ca257" exitCode=0
Jul 24 03:58:48.588696 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:48.588656    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r" event={"ID":"063c0851-b09e-4695-85d5-c7f0543c8b6f","Type":"ContainerDied","Data":"9c10af393728dbfe0937c964026bf55c939660ba0df15fe04347d8a8133ca257"}
Jul 24 03:58:49.727202 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:49.727174    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r"
Jul 24 03:58:49.864733 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:49.864635    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/063c0851-b09e-4695-85d5-c7f0543c8b6f-maas-api-service-ca\") pod \"063c0851-b09e-4695-85d5-c7f0543c8b6f\" (UID: \"063c0851-b09e-4695-85d5-c7f0543c8b6f\") "
Jul 24 03:58:49.865005 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:49.864981    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/063c0851-b09e-4695-85d5-c7f0543c8b6f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "063c0851-b09e-4695-85d5-c7f0543c8b6f" (UID: "063c0851-b09e-4695-85d5-c7f0543c8b6f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:58:49.965337 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:49.965283    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/063c0851-b09e-4695-85d5-c7f0543c8b6f-maas-api-service-ca\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:58:50.599820 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:50.599780    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r" event={"ID":"063c0851-b09e-4695-85d5-c7f0543c8b6f","Type":"ContainerDied","Data":"b356fdedd7d7adbf7b25d26cfb78b7542d53245e80a37d7ebfdd5bc470f0999b"}
Jul 24 03:58:50.599820 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:50.599807    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r"
Jul 24 03:58:50.599820 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:50.599831    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b356fdedd7d7adbf7b25d26cfb78b7542d53245e80a37d7ebfdd5bc470f0999b"
Jul 24 03:58:57.494456 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.494391    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"]
Jul 24 03:58:57.494957 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.494740    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7" podUID="1d4348ef-f155-49ea-90b4-542569afa11f" containerName="payload-processing" containerID="cri-o://5df0998967340ed5220ccb9055ca87c745df77b7a7e660cfb8d0901d240a5e1e" gracePeriod=30
Jul 24 03:58:57.503263 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.503229    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"]
Jul 24 03:58:57.503619 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.503568    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn" podUID="ee570b27-a617-43d8-bfdb-787ad21a020a" containerName="payload-pre-processing" containerID="cri-o://2ef772c79cfab73e52e499650f33e34a4c9fd88336178c4721b18381c6d39edf" gracePeriod=30
Jul 24 03:58:57.637319 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.637274    2574 generic.go:356] "Generic (PLEG): container finished" podID="1d4348ef-f155-49ea-90b4-542569afa11f" containerID="5df0998967340ed5220ccb9055ca87c745df77b7a7e660cfb8d0901d240a5e1e" exitCode=0
Jul 24 03:58:57.637535 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.637346    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7" event={"ID":"1d4348ef-f155-49ea-90b4-542569afa11f","Type":"ContainerDied","Data":"5df0998967340ed5220ccb9055ca87c745df77b7a7e660cfb8d0901d240a5e1e"}
Jul 24 03:58:57.639357 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.639329    2574 generic.go:356] "Generic (PLEG): container finished" podID="ee570b27-a617-43d8-bfdb-787ad21a020a" containerID="2ef772c79cfab73e52e499650f33e34a4c9fd88336178c4721b18381c6d39edf" exitCode=0
Jul 24 03:58:57.639539 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.639378    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn" event={"ID":"ee570b27-a617-43d8-bfdb-787ad21a020a","Type":"ContainerDied","Data":"2ef772c79cfab73e52e499650f33e34a4c9fd88336178c4721b18381c6d39edf"}
Jul 24 03:58:57.789654 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.789628    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"
Jul 24 03:58:57.792529 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.792507    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"
Jul 24 03:58:57.943993 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.943965    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qnrq\" (UniqueName: \"kubernetes.io/projected/ee570b27-a617-43d8-bfdb-787ad21a020a-kube-api-access-9qnrq\") pod \"ee570b27-a617-43d8-bfdb-787ad21a020a\" (UID: \"ee570b27-a617-43d8-bfdb-787ad21a020a\") "
Jul 24 03:58:57.944211 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.944042    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee570b27-a617-43d8-bfdb-787ad21a020a-plugins-config-volume\") pod \"ee570b27-a617-43d8-bfdb-787ad21a020a\" (UID: \"ee570b27-a617-43d8-bfdb-787ad21a020a\") "
Jul 24 03:58:57.944211 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.944067    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d4348ef-f155-49ea-90b4-542569afa11f-plugins-config-volume\") pod \"1d4348ef-f155-49ea-90b4-542569afa11f\" (UID: \"1d4348ef-f155-49ea-90b4-542569afa11f\") "
Jul 24 03:58:57.944211 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.944154    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsmzt\" (UniqueName: \"kubernetes.io/projected/1d4348ef-f155-49ea-90b4-542569afa11f-kube-api-access-nsmzt\") pod \"1d4348ef-f155-49ea-90b4-542569afa11f\" (UID: \"1d4348ef-f155-49ea-90b4-542569afa11f\") "
Jul 24 03:58:57.944557 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.944529    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d4348ef-f155-49ea-90b4-542569afa11f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1d4348ef-f155-49ea-90b4-542569afa11f" (UID: "1d4348ef-f155-49ea-90b4-542569afa11f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:58:57.944668 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.944524    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee570b27-a617-43d8-bfdb-787ad21a020a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ee570b27-a617-43d8-bfdb-787ad21a020a" (UID: "ee570b27-a617-43d8-bfdb-787ad21a020a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:58:57.946504 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.946454    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee570b27-a617-43d8-bfdb-787ad21a020a-kube-api-access-9qnrq" (OuterVolumeSpecName: "kube-api-access-9qnrq") pod "ee570b27-a617-43d8-bfdb-787ad21a020a" (UID: "ee570b27-a617-43d8-bfdb-787ad21a020a"). InnerVolumeSpecName "kube-api-access-9qnrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:58:57.946598 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:57.946539    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d4348ef-f155-49ea-90b4-542569afa11f-kube-api-access-nsmzt" (OuterVolumeSpecName: "kube-api-access-nsmzt") pod "1d4348ef-f155-49ea-90b4-542569afa11f" (UID: "1d4348ef-f155-49ea-90b4-542569afa11f"). InnerVolumeSpecName "kube-api-access-nsmzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:58:58.045323 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.045231    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee570b27-a617-43d8-bfdb-787ad21a020a-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:58:58.045323 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.045265    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d4348ef-f155-49ea-90b4-542569afa11f-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:58:58.045323 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.045274    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsmzt\" (UniqueName: \"kubernetes.io/projected/1d4348ef-f155-49ea-90b4-542569afa11f-kube-api-access-nsmzt\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:58:58.045323 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.045283    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qnrq\" (UniqueName: \"kubernetes.io/projected/ee570b27-a617-43d8-bfdb-787ad21a020a-kube-api-access-9qnrq\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:58:58.646291 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.646255    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"
Jul 24 03:58:58.646291 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.646276    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn" event={"ID":"ee570b27-a617-43d8-bfdb-787ad21a020a","Type":"ContainerDied","Data":"7ff83f2e96baa0448edb14daae65568cf86edd2d56b3437a4fd3c7f34a100bbb"}
Jul 24 03:58:58.646835 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.646356    2574 scope.go:117] "RemoveContainer" containerID="2ef772c79cfab73e52e499650f33e34a4c9fd88336178c4721b18381c6d39edf"
Jul 24 03:58:58.647806 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.647665    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7" event={"ID":"1d4348ef-f155-49ea-90b4-542569afa11f","Type":"ContainerDied","Data":"f977d7b03ae5c15d66011e7b91a6d5dbf35e6779644dadbadaf1cd2c7711d2e7"}
Jul 24 03:58:58.647806 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.647686    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"
Jul 24 03:58:58.655204 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.655185    2574 scope.go:117] "RemoveContainer" containerID="5df0998967340ed5220ccb9055ca87c745df77b7a7e660cfb8d0901d240a5e1e"
Jul 24 03:58:58.673513 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.673487    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"]
Jul 24 03:58:58.677546 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.677521    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-b0d347-774849d9fb-25mp7"]
Jul 24 03:58:58.687131 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.687104    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"]
Jul 24 03:58:58.689298 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:58.689279    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-b0d347-856d599c5b-ztptn"]
Jul 24 03:58:59.574137 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:59.574100    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d4348ef-f155-49ea-90b4-542569afa11f" path="/var/lib/kubelet/pods/1d4348ef-f155-49ea-90b4-542569afa11f/volumes"
Jul 24 03:58:59.574499 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:58:59.574484    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee570b27-a617-43d8-bfdb-787ad21a020a" path="/var/lib/kubelet/pods/ee570b27-a617-43d8-bfdb-787ad21a020a/volumes"
Jul 24 03:59:02.128704 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:02.128654    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r"]
Jul 24 03:59:02.134937 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:02.134890    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b0d347-7mx6r"]
Jul 24 03:59:03.573146 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:03.573110    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="063c0851-b09e-4695-85d5-c7f0543c8b6f" path="/var/lib/kubelet/pods/063c0851-b09e-4695-85d5-c7f0543c8b6f/volumes"
Jul 24 03:59:14.006959 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.006924    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"]
Jul 24 03:59:14.007534 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.007358    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4348ef-f155-49ea-90b4-542569afa11f" containerName="payload-processing"
Jul 24 03:59:14.007534 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.007371    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4348ef-f155-49ea-90b4-542569afa11f" containerName="payload-processing"
Jul 24 03:59:14.007534 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.007379    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="063c0851-b09e-4695-85d5-c7f0543c8b6f" containerName="revoke-keys"
Jul 24 03:59:14.007534 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.007385    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="063c0851-b09e-4695-85d5-c7f0543c8b6f" containerName="revoke-keys"
Jul 24 03:59:14.007534 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.007395    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee570b27-a617-43d8-bfdb-787ad21a020a" containerName="payload-pre-processing"
Jul 24 03:59:14.007534 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.007400    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee570b27-a617-43d8-bfdb-787ad21a020a" containerName="payload-pre-processing"
Jul 24 03:59:14.007534 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.007501    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4348ef-f155-49ea-90b4-542569afa11f" containerName="payload-processing"
Jul 24 03:59:14.007534 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.007512    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee570b27-a617-43d8-bfdb-787ad21a020a" containerName="payload-pre-processing"
Jul 24 03:59:14.007534 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.007520    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="063c0851-b09e-4695-85d5-c7f0543c8b6f" containerName="revoke-keys"
Jul 24 03:59:14.010966 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.010948    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"
Jul 24 03:59:14.013293 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.013268    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-f822cd-dockercfg-g5sz8\""
Jul 24 03:59:14.013293 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.013283    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-f822cd\""
Jul 24 03:59:14.025552 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.025525    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"]
Jul 24 03:59:14.034302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.033521    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"]
Jul 24 03:59:14.034302 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.033658    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"
Jul 24 03:59:14.045489 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.045458    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"]
Jul 24 03:59:14.094691 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.094654    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6b05c2e-b482-4d6a-89e5-c2b504ae6412-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4\" (UID: \"f6b05c2e-b482-4d6a-89e5-c2b504ae6412\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"
Jul 24 03:59:14.094881 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.094705    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss782\" (UniqueName: \"kubernetes.io/projected/f6b05c2e-b482-4d6a-89e5-c2b504ae6412-kube-api-access-ss782\") pod \"payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4\" (UID: \"f6b05c2e-b482-4d6a-89e5-c2b504ae6412\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"
Jul 24 03:59:14.094955 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.094875    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99fd500b-c91a-4ec5-b0da-6340caadb13b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm\" (UID: \"99fd500b-c91a-4ec5-b0da-6340caadb13b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"
Jul 24 03:59:14.094955 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.094919    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plpcd\" (UniqueName: \"kubernetes.io/projected/99fd500b-c91a-4ec5-b0da-6340caadb13b-kube-api-access-plpcd\") pod \"payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm\" (UID: \"99fd500b-c91a-4ec5-b0da-6340caadb13b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"
Jul 24 03:59:14.195891 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.195836    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6b05c2e-b482-4d6a-89e5-c2b504ae6412-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4\" (UID: \"f6b05c2e-b482-4d6a-89e5-c2b504ae6412\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"
Jul 24 03:59:14.195891 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.195901    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss782\" (UniqueName: \"kubernetes.io/projected/f6b05c2e-b482-4d6a-89e5-c2b504ae6412-kube-api-access-ss782\") pod \"payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4\" (UID: \"f6b05c2e-b482-4d6a-89e5-c2b504ae6412\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"
Jul 24 03:59:14.196135 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.195969    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99fd500b-c91a-4ec5-b0da-6340caadb13b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm\" (UID: \"99fd500b-c91a-4ec5-b0da-6340caadb13b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"
Jul 24 03:59:14.196135 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.195990    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plpcd\" (UniqueName: \"kubernetes.io/projected/99fd500b-c91a-4ec5-b0da-6340caadb13b-kube-api-access-plpcd\") pod \"payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm\" (UID: \"99fd500b-c91a-4ec5-b0da-6340caadb13b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"
Jul 24 03:59:14.196618 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.196598    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6b05c2e-b482-4d6a-89e5-c2b504ae6412-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4\" (UID: \"f6b05c2e-b482-4d6a-89e5-c2b504ae6412\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"
Jul 24 03:59:14.196746 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.196727    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99fd500b-c91a-4ec5-b0da-6340caadb13b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm\" (UID: \"99fd500b-c91a-4ec5-b0da-6340caadb13b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"
Jul 24 03:59:14.209029 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.208998    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss782\" (UniqueName: \"kubernetes.io/projected/f6b05c2e-b482-4d6a-89e5-c2b504ae6412-kube-api-access-ss782\") pod \"payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4\" (UID: \"f6b05c2e-b482-4d6a-89e5-c2b504ae6412\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"
Jul 24 03:59:14.209029 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.209004    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plpcd\" (UniqueName: \"kubernetes.io/projected/99fd500b-c91a-4ec5-b0da-6340caadb13b-kube-api-access-plpcd\") pod \"payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm\" (UID: \"99fd500b-c91a-4ec5-b0da-6340caadb13b\") " pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"
Jul 24 03:59:14.321281 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.321187    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"
Jul 24 03:59:14.348841 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.348806    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"
Jul 24 03:59:14.500623 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.500591    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"]
Jul 24 03:59:14.502773 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:59:14.502734    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6b05c2e_b482_4d6a_89e5_c2b504ae6412.slice/crio-a195b7b492e901fb3a35d0d0ada20bc39400051798335ad521458900f107c6c5 WatchSource:0}: Error finding container a195b7b492e901fb3a35d0d0ada20bc39400051798335ad521458900f107c6c5: Status 404 returned error can't find the container with id a195b7b492e901fb3a35d0d0ada20bc39400051798335ad521458900f107c6c5
Jul 24 03:59:14.531059 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.531036    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"]
Jul 24 03:59:14.533089 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:59:14.533061    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99fd500b_c91a_4ec5_b0da_6340caadb13b.slice/crio-4ffcef9d4a6109f807ec9408ccd19c299ba7b3998b892be332744d9bb0f4f7aa WatchSource:0}: Error finding container 4ffcef9d4a6109f807ec9408ccd19c299ba7b3998b892be332744d9bb0f4f7aa: Status 404 returned error can't find the container with id 4ffcef9d4a6109f807ec9408ccd19c299ba7b3998b892be332744d9bb0f4f7aa
Jul 24 03:59:14.736585 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.736534    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm" event={"ID":"99fd500b-c91a-4ec5-b0da-6340caadb13b","Type":"ContainerStarted","Data":"e2f1e17a81b362b4f670cd39b2fe2b6862c26a1ecfc2ea907bc55a006b41c0ea"}
Jul 24 03:59:14.736856 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.736838    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm" event={"ID":"99fd500b-c91a-4ec5-b0da-6340caadb13b","Type":"ContainerStarted","Data":"4ffcef9d4a6109f807ec9408ccd19c299ba7b3998b892be332744d9bb0f4f7aa"}
Jul 24 03:59:14.736976 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.736950    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"
Jul 24 03:59:14.738411 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.738383    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4" event={"ID":"f6b05c2e-b482-4d6a-89e5-c2b504ae6412","Type":"ContainerStarted","Data":"ffa80cb631f957eadd6dd4e66b032a66320360fd83e302489937ed2327ca36c3"}
Jul 24 03:59:14.738550 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.738462    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4" event={"ID":"f6b05c2e-b482-4d6a-89e5-c2b504ae6412","Type":"ContainerStarted","Data":"a195b7b492e901fb3a35d0d0ada20bc39400051798335ad521458900f107c6c5"}
Jul 24 03:59:14.738630 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.738561    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"
Jul 24 03:59:14.778094 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:14.778043    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm" podStartSLOduration=1.778028041 podStartE2EDuration="1.778028041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:59:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:59:14.777610126 +0000 UTC m=+2509.740703425" watchObservedRunningTime="2026-07-24 03:59:14.778028041 +0000 UTC m=+2509.741121327"
Jul 24 03:59:22.116405 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.116297    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4" podStartSLOduration=9.116278146 podStartE2EDuration="9.116278146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:59:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:59:14.799849284 +0000 UTC m=+2509.762942571" watchObservedRunningTime="2026-07-24 03:59:22.116278146 +0000 UTC m=+2517.079371446"
Jul 24 03:59:22.117665 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.117625    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"]
Jul 24 03:59:22.122318 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.122297    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"
Jul 24 03:59:22.124844 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.124822    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-b23325\""
Jul 24 03:59:22.124963 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.124909    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-b23325-dockercfg-qh2pd\""
Jul 24 03:59:22.133175 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.133145    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"]
Jul 24 03:59:22.170041 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.170001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae8819bf-5272-4ef9-8441-f8901f6bb75a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn\" (UID: \"ae8819bf-5272-4ef9-8441-f8901f6bb75a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"
Jul 24 03:59:22.170240 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.170114    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrddd\" (UniqueName: \"kubernetes.io/projected/ae8819bf-5272-4ef9-8441-f8901f6bb75a-kube-api-access-rrddd\") pod \"payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn\" (UID: \"ae8819bf-5272-4ef9-8441-f8901f6bb75a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"
Jul 24 03:59:22.203972 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.203934    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"]
Jul 24 03:59:22.207486 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.207462    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"
Jul 24 03:59:22.220525 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.220496    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"]
Jul 24 03:59:22.271701 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.271657    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae8819bf-5272-4ef9-8441-f8901f6bb75a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn\" (UID: \"ae8819bf-5272-4ef9-8441-f8901f6bb75a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"
Jul 24 03:59:22.271921 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.271717    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4f1957-1ffd-4cac-82dd-b66cc6a3b537-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng\" (UID: \"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537\") " pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"
Jul 24 03:59:22.271921 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.271826    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrddd\" (UniqueName: \"kubernetes.io/projected/ae8819bf-5272-4ef9-8441-f8901f6bb75a-kube-api-access-rrddd\") pod \"payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn\" (UID: \"ae8819bf-5272-4ef9-8441-f8901f6bb75a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"
Jul 24 03:59:22.271921 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.271903    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfqtz\" (UniqueName: \"kubernetes.io/projected/2d4f1957-1ffd-4cac-82dd-b66cc6a3b537-kube-api-access-vfqtz\") pod \"payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng\" (UID: \"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537\") " pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"
Jul 24 03:59:22.272465 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.272446    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae8819bf-5272-4ef9-8441-f8901f6bb75a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn\" (UID: \"ae8819bf-5272-4ef9-8441-f8901f6bb75a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"
Jul 24 03:59:22.280504 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.280483    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrddd\" (UniqueName: \"kubernetes.io/projected/ae8819bf-5272-4ef9-8441-f8901f6bb75a-kube-api-access-rrddd\") pod \"payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn\" (UID: \"ae8819bf-5272-4ef9-8441-f8901f6bb75a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"
Jul 24 03:59:22.372484 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.372372    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4f1957-1ffd-4cac-82dd-b66cc6a3b537-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng\" (UID: \"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537\") " pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"
Jul 24 03:59:22.372631 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.372495    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfqtz\" (UniqueName: \"kubernetes.io/projected/2d4f1957-1ffd-4cac-82dd-b66cc6a3b537-kube-api-access-vfqtz\") pod \"payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng\" (UID: \"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537\") " pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"
Jul 24 03:59:22.373100 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.373079    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4f1957-1ffd-4cac-82dd-b66cc6a3b537-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng\" (UID: \"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537\") " pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"
Jul 24 03:59:22.381378 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.381351    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfqtz\" (UniqueName: \"kubernetes.io/projected/2d4f1957-1ffd-4cac-82dd-b66cc6a3b537-kube-api-access-vfqtz\") pod \"payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng\" (UID: \"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537\") " pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"
Jul 24 03:59:22.434803 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.434752    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"
Jul 24 03:59:22.518784 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.518751    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"
Jul 24 03:59:22.594448 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.594360    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"]
Jul 24 03:59:22.596809 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:59:22.596774    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae8819bf_5272_4ef9_8441_f8901f6bb75a.slice/crio-8c419671047712e7e451fe30420f1f3783e30eda5fcae97c03dcd780860c2c41 WatchSource:0}: Error finding container 8c419671047712e7e451fe30420f1f3783e30eda5fcae97c03dcd780860c2c41: Status 404 returned error can't find the container with id 8c419671047712e7e451fe30420f1f3783e30eda5fcae97c03dcd780860c2c41
Jul 24 03:59:22.677959 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.677870    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"]
Jul 24 03:59:22.680203 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:59:22.680178    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d4f1957_1ffd_4cac_82dd_b66cc6a3b537.slice/crio-c723066d0b0824e0aff4a90f0e0ad3641218408a059b96a8af57375c31a759d8 WatchSource:0}: Error finding container c723066d0b0824e0aff4a90f0e0ad3641218408a059b96a8af57375c31a759d8: Status 404 returned error can't find the container with id c723066d0b0824e0aff4a90f0e0ad3641218408a059b96a8af57375c31a759d8
Jul 24 03:59:22.782644 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.782607    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng" event={"ID":"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537","Type":"ContainerStarted","Data":"35fc99d8ce12112e4a48f521bd302563f350bca8b9dcf9352c5caef584eca2ee"}
Jul 24 03:59:22.782837 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.782664    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng" event={"ID":"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537","Type":"ContainerStarted","Data":"c723066d0b0824e0aff4a90f0e0ad3641218408a059b96a8af57375c31a759d8"}
Jul 24 03:59:22.782837 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.782783    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"
Jul 24 03:59:22.784299 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.784278    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn" event={"ID":"ae8819bf-5272-4ef9-8441-f8901f6bb75a","Type":"ContainerStarted","Data":"59cc589354b7d00d7533a9ad68e0e347e6001d602253d3dc625cf65e85df6d5a"}
Jul 24 03:59:22.784447 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.784314    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn" event={"ID":"ae8819bf-5272-4ef9-8441-f8901f6bb75a","Type":"ContainerStarted","Data":"8c419671047712e7e451fe30420f1f3783e30eda5fcae97c03dcd780860c2c41"}
Jul 24 03:59:22.784528 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.784463    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"
Jul 24 03:59:22.803889 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.803839    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng" podStartSLOduration=0.803823829 podStartE2EDuration="803.823829ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:59:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:59:22.803222243 +0000 UTC m=+2517.766315535" watchObservedRunningTime="2026-07-24 03:59:22.803823829 +0000 UTC m=+2517.766917116"
Jul 24 03:59:22.823132 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:22.823080    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn" podStartSLOduration=0.82306501 podStartE2EDuration="823.06501ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:59:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:59:22.822516291 +0000 UTC m=+2517.785609579" watchObservedRunningTime="2026-07-24 03:59:22.82306501 +0000 UTC m=+2517.786158297"
Jul 24 03:59:25.745887 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:25.745852    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"
Jul 24 03:59:25.746390 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:25.745980    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"
Jul 24 03:59:25.795479 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:25.795410    2574 scope.go:117] "RemoveContainer" containerID="e5417dd547c6dc84cbf37421341c8e867779867e49ca6e115a1485b3d3433853"
Jul 24 03:59:33.791655 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:33.791624    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"
Jul 24 03:59:33.792176 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:33.792159    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"
Jul 24 03:59:33.860916 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:33.860873    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf"]
Jul 24 03:59:33.864365 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:33.864339    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf"
Jul 24 03:59:33.867025 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:33.867001    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 03:59:33.867177 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:33.867080    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 03:59:33.888854 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:33.888814    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf"]
Jul 24 03:59:33.993965 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:33.993929    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/204714fc-c6c4-4821-8c98-a7460196f1c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf\" (UID: \"204714fc-c6c4-4821-8c98-a7460196f1c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf"
Jul 24 03:59:34.094948 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:34.094857    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/204714fc-c6c4-4821-8c98-a7460196f1c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf\" (UID: \"204714fc-c6c4-4821-8c98-a7460196f1c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf"
Jul 24 03:59:34.095544 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:34.095523    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/204714fc-c6c4-4821-8c98-a7460196f1c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf\" (UID: \"204714fc-c6c4-4821-8c98-a7460196f1c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf"
Jul 24 03:59:34.175525 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:34.175487    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf"
Jul 24 03:59:34.316111 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:34.316084    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf"]
Jul 24 03:59:34.318199 ip-10-0-140-227 kubenswrapper[2574]: W0724 03:59:34.318164    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod204714fc_c6c4_4821_8c98_a7460196f1c8.slice/crio-5800ce1bfe3e0130e96f59c2263e04fb9013c12f4c9a689e49d458fe3959c17d WatchSource:0}: Error finding container 5800ce1bfe3e0130e96f59c2263e04fb9013c12f4c9a689e49d458fe3959c17d: Status 404 returned error can't find the container with id 5800ce1bfe3e0130e96f59c2263e04fb9013c12f4c9a689e49d458fe3959c17d
Jul 24 03:59:34.852068 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:34.852032    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf" event={"ID":"204714fc-c6c4-4821-8c98-a7460196f1c8","Type":"ContainerStarted","Data":"79aa186fa971a9a2532d7d484bea474fda450b193abf9aca345b9798bd65437f"}
Jul 24 03:59:34.852553 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:34.852103    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf" event={"ID":"204714fc-c6c4-4821-8c98-a7460196f1c8","Type":"ContainerStarted","Data":"5800ce1bfe3e0130e96f59c2263e04fb9013c12f4c9a689e49d458fe3959c17d"}
Jul 24 03:59:34.866997 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:34.866941    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf" podStartSLOduration=1.8669233840000001 podStartE2EDuration="1.866923384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 03:59:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 03:59:34.866229101 +0000 UTC m=+2529.829322389" watchObservedRunningTime="2026-07-24 03:59:34.866923384 +0000 UTC m=+2529.830016671"
Jul 24 03:59:35.858091 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:35.858053    2574 generic.go:356] "Generic (PLEG): container finished" podID="204714fc-c6c4-4821-8c98-a7460196f1c8" containerID="79aa186fa971a9a2532d7d484bea474fda450b193abf9aca345b9798bd65437f" exitCode=0
Jul 24 03:59:35.858481 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:35.858137    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf" event={"ID":"204714fc-c6c4-4821-8c98-a7460196f1c8","Type":"ContainerDied","Data":"79aa186fa971a9a2532d7d484bea474fda450b193abf9aca345b9798bd65437f"}
Jul 24 03:59:36.999080 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:36.999053    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf"
Jul 24 03:59:37.126867 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:37.126777    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/204714fc-c6c4-4821-8c98-a7460196f1c8-maas-api-service-ca\") pod \"204714fc-c6c4-4821-8c98-a7460196f1c8\" (UID: \"204714fc-c6c4-4821-8c98-a7460196f1c8\") "
Jul 24 03:59:37.127178 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:37.127158    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/204714fc-c6c4-4821-8c98-a7460196f1c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "204714fc-c6c4-4821-8c98-a7460196f1c8" (UID: "204714fc-c6c4-4821-8c98-a7460196f1c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:59:37.228441 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:37.228405    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/204714fc-c6c4-4821-8c98-a7460196f1c8-maas-api-service-ca\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:59:37.870168 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:37.870062    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf" event={"ID":"204714fc-c6c4-4821-8c98-a7460196f1c8","Type":"ContainerDied","Data":"5800ce1bfe3e0130e96f59c2263e04fb9013c12f4c9a689e49d458fe3959c17d"}
Jul 24 03:59:37.870168 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:37.870085    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf"
Jul 24 03:59:37.870168 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:37.870122    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5800ce1bfe3e0130e96f59c2263e04fb9013c12f4c9a689e49d458fe3959c17d"
Jul 24 03:59:49.589842 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.589806    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"]
Jul 24 03:59:49.590470 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.590063    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm" podUID="99fd500b-c91a-4ec5-b0da-6340caadb13b" containerName="payload-processing" containerID="cri-o://e2f1e17a81b362b4f670cd39b2fe2b6862c26a1ecfc2ea907bc55a006b41c0ea" gracePeriod=30
Jul 24 03:59:49.601982 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.601951    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"]
Jul 24 03:59:49.602240 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.602214    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4" podUID="f6b05c2e-b482-4d6a-89e5-c2b504ae6412" containerName="payload-pre-processing" containerID="cri-o://ffa80cb631f957eadd6dd4e66b032a66320360fd83e302489937ed2327ca36c3" gracePeriod=30
Jul 24 03:59:49.859610 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.859576    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"
Jul 24 03:59:49.882221 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.882195    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"
Jul 24 03:59:49.931936 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.931905    2574 generic.go:356] "Generic (PLEG): container finished" podID="99fd500b-c91a-4ec5-b0da-6340caadb13b" containerID="e2f1e17a81b362b4f670cd39b2fe2b6862c26a1ecfc2ea907bc55a006b41c0ea" exitCode=0
Jul 24 03:59:49.932136 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.931961    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"
Jul 24 03:59:49.932136 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.931990    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm" event={"ID":"99fd500b-c91a-4ec5-b0da-6340caadb13b","Type":"ContainerDied","Data":"e2f1e17a81b362b4f670cd39b2fe2b6862c26a1ecfc2ea907bc55a006b41c0ea"}
Jul 24 03:59:49.932136 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.932041    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm" event={"ID":"99fd500b-c91a-4ec5-b0da-6340caadb13b","Type":"ContainerDied","Data":"4ffcef9d4a6109f807ec9408ccd19c299ba7b3998b892be332744d9bb0f4f7aa"}
Jul 24 03:59:49.932136 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.932074    2574 scope.go:117] "RemoveContainer" containerID="e2f1e17a81b362b4f670cd39b2fe2b6862c26a1ecfc2ea907bc55a006b41c0ea"
Jul 24 03:59:49.933367 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.933343    2574 generic.go:356] "Generic (PLEG): container finished" podID="f6b05c2e-b482-4d6a-89e5-c2b504ae6412" containerID="ffa80cb631f957eadd6dd4e66b032a66320360fd83e302489937ed2327ca36c3" exitCode=0
Jul 24 03:59:49.933504 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.933386    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"
Jul 24 03:59:49.933504 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.933403    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4" event={"ID":"f6b05c2e-b482-4d6a-89e5-c2b504ae6412","Type":"ContainerDied","Data":"ffa80cb631f957eadd6dd4e66b032a66320360fd83e302489937ed2327ca36c3"}
Jul 24 03:59:49.933504 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.933458    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4" event={"ID":"f6b05c2e-b482-4d6a-89e5-c2b504ae6412","Type":"ContainerDied","Data":"a195b7b492e901fb3a35d0d0ada20bc39400051798335ad521458900f107c6c5"}
Jul 24 03:59:49.941616 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.941597    2574 scope.go:117] "RemoveContainer" containerID="e2f1e17a81b362b4f670cd39b2fe2b6862c26a1ecfc2ea907bc55a006b41c0ea"
Jul 24 03:59:49.941887 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:59:49.941865    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2f1e17a81b362b4f670cd39b2fe2b6862c26a1ecfc2ea907bc55a006b41c0ea\": container with ID starting with e2f1e17a81b362b4f670cd39b2fe2b6862c26a1ecfc2ea907bc55a006b41c0ea not found: ID does not exist" containerID="e2f1e17a81b362b4f670cd39b2fe2b6862c26a1ecfc2ea907bc55a006b41c0ea"
Jul 24 03:59:49.941996 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.941895    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f1e17a81b362b4f670cd39b2fe2b6862c26a1ecfc2ea907bc55a006b41c0ea"} err="failed to get container status \"e2f1e17a81b362b4f670cd39b2fe2b6862c26a1ecfc2ea907bc55a006b41c0ea\": rpc error: code = NotFound desc = could not find container \"e2f1e17a81b362b4f670cd39b2fe2b6862c26a1ecfc2ea907bc55a006b41c0ea\": container with ID starting with e2f1e17a81b362b4f670cd39b2fe2b6862c26a1ecfc2ea907bc55a006b41c0ea not found: ID does not exist"
Jul 24 03:59:49.941996 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.941914    2574 scope.go:117] "RemoveContainer" containerID="ffa80cb631f957eadd6dd4e66b032a66320360fd83e302489937ed2327ca36c3"
Jul 24 03:59:49.947183 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.947159    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99fd500b-c91a-4ec5-b0da-6340caadb13b-plugins-config-volume\") pod \"99fd500b-c91a-4ec5-b0da-6340caadb13b\" (UID: \"99fd500b-c91a-4ec5-b0da-6340caadb13b\") "
Jul 24 03:59:49.947289 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.947225    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss782\" (UniqueName: \"kubernetes.io/projected/f6b05c2e-b482-4d6a-89e5-c2b504ae6412-kube-api-access-ss782\") pod \"f6b05c2e-b482-4d6a-89e5-c2b504ae6412\" (UID: \"f6b05c2e-b482-4d6a-89e5-c2b504ae6412\") "
Jul 24 03:59:49.947355 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.947297    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6b05c2e-b482-4d6a-89e5-c2b504ae6412-plugins-config-volume\") pod \"f6b05c2e-b482-4d6a-89e5-c2b504ae6412\" (UID: \"f6b05c2e-b482-4d6a-89e5-c2b504ae6412\") "
Jul 24 03:59:49.947416 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.947355    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plpcd\" (UniqueName: \"kubernetes.io/projected/99fd500b-c91a-4ec5-b0da-6340caadb13b-kube-api-access-plpcd\") pod \"99fd500b-c91a-4ec5-b0da-6340caadb13b\" (UID: \"99fd500b-c91a-4ec5-b0da-6340caadb13b\") "
Jul 24 03:59:49.947743 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.947668    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99fd500b-c91a-4ec5-b0da-6340caadb13b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "99fd500b-c91a-4ec5-b0da-6340caadb13b" (UID: "99fd500b-c91a-4ec5-b0da-6340caadb13b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:59:49.947743 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.947713    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6b05c2e-b482-4d6a-89e5-c2b504ae6412-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f6b05c2e-b482-4d6a-89e5-c2b504ae6412" (UID: "f6b05c2e-b482-4d6a-89e5-c2b504ae6412"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 03:59:49.949555 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.949530    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6b05c2e-b482-4d6a-89e5-c2b504ae6412-kube-api-access-ss782" (OuterVolumeSpecName: "kube-api-access-ss782") pod "f6b05c2e-b482-4d6a-89e5-c2b504ae6412" (UID: "f6b05c2e-b482-4d6a-89e5-c2b504ae6412"). InnerVolumeSpecName "kube-api-access-ss782". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:59:49.949946 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.949918    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99fd500b-c91a-4ec5-b0da-6340caadb13b-kube-api-access-plpcd" (OuterVolumeSpecName: "kube-api-access-plpcd") pod "99fd500b-c91a-4ec5-b0da-6340caadb13b" (UID: "99fd500b-c91a-4ec5-b0da-6340caadb13b"). InnerVolumeSpecName "kube-api-access-plpcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 03:59:49.951566 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.951549    2574 scope.go:117] "RemoveContainer" containerID="ffa80cb631f957eadd6dd4e66b032a66320360fd83e302489937ed2327ca36c3"
Jul 24 03:59:49.951871 ip-10-0-140-227 kubenswrapper[2574]: E0724 03:59:49.951851    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffa80cb631f957eadd6dd4e66b032a66320360fd83e302489937ed2327ca36c3\": container with ID starting with ffa80cb631f957eadd6dd4e66b032a66320360fd83e302489937ed2327ca36c3 not found: ID does not exist" containerID="ffa80cb631f957eadd6dd4e66b032a66320360fd83e302489937ed2327ca36c3"
Jul 24 03:59:49.951913 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:49.951880    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffa80cb631f957eadd6dd4e66b032a66320360fd83e302489937ed2327ca36c3"} err="failed to get container status \"ffa80cb631f957eadd6dd4e66b032a66320360fd83e302489937ed2327ca36c3\": rpc error: code = NotFound desc = could not find container \"ffa80cb631f957eadd6dd4e66b032a66320360fd83e302489937ed2327ca36c3\": container with ID starting with ffa80cb631f957eadd6dd4e66b032a66320360fd83e302489937ed2327ca36c3 not found: ID does not exist"
Jul 24 03:59:50.048772 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:50.048737    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss782\" (UniqueName: \"kubernetes.io/projected/f6b05c2e-b482-4d6a-89e5-c2b504ae6412-kube-api-access-ss782\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:59:50.048772 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:50.048765    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6b05c2e-b482-4d6a-89e5-c2b504ae6412-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:59:50.048772 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:50.048775    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plpcd\" (UniqueName: \"kubernetes.io/projected/99fd500b-c91a-4ec5-b0da-6340caadb13b-kube-api-access-plpcd\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:59:50.048772 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:50.048784    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99fd500b-c91a-4ec5-b0da-6340caadb13b-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 03:59:50.260308 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:50.260277    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"]
Jul 24 03:59:50.262137 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:50.262114    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f822cd-59c8f95888-9v4j4"]
Jul 24 03:59:50.272708 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:50.272670    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"]
Jul 24 03:59:50.279224 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:50.279195    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f822cd-846f68546b-jkcwm"]
Jul 24 03:59:51.573809 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:51.573771    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99fd500b-c91a-4ec5-b0da-6340caadb13b" path="/var/lib/kubelet/pods/99fd500b-c91a-4ec5-b0da-6340caadb13b/volumes"
Jul 24 03:59:51.574187 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:51.574123    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6b05c2e-b482-4d6a-89e5-c2b504ae6412" path="/var/lib/kubelet/pods/f6b05c2e-b482-4d6a-89e5-c2b504ae6412/volumes"
Jul 24 03:59:53.920888 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:53.920849    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf"]
Jul 24 03:59:53.924290 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:53.924265    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f822cd-tn7xf"]
Jul 24 03:59:55.573973 ip-10-0-140-227 kubenswrapper[2574]: I0724 03:59:55.573929    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="204714fc-c6c4-4821-8c98-a7460196f1c8" path="/var/lib/kubelet/pods/204714fc-c6c4-4821-8c98-a7460196f1c8/volumes"
Jul 24 04:00:00.155593 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.155561    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747760-b7zw9"]
Jul 24 04:00:00.155989 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.155942    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6b05c2e-b482-4d6a-89e5-c2b504ae6412" containerName="payload-pre-processing"
Jul 24 04:00:00.155989 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.155955    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6b05c2e-b482-4d6a-89e5-c2b504ae6412" containerName="payload-pre-processing"
Jul 24 04:00:00.155989 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.155967    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99fd500b-c91a-4ec5-b0da-6340caadb13b" containerName="payload-processing"
Jul 24 04:00:00.155989 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.155972    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="99fd500b-c91a-4ec5-b0da-6340caadb13b" containerName="payload-processing"
Jul 24 04:00:00.156215 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.155995    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="204714fc-c6c4-4821-8c98-a7460196f1c8" containerName="revoke-keys"
Jul 24 04:00:00.156215 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.156001    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="204714fc-c6c4-4821-8c98-a7460196f1c8" containerName="revoke-keys"
Jul 24 04:00:00.156215 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.156056    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="204714fc-c6c4-4821-8c98-a7460196f1c8" containerName="revoke-keys"
Jul 24 04:00:00.156215 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.156065    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6b05c2e-b482-4d6a-89e5-c2b504ae6412" containerName="payload-pre-processing"
Jul 24 04:00:00.156215 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.156070    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="99fd500b-c91a-4ec5-b0da-6340caadb13b" containerName="payload-processing"
Jul 24 04:00:00.161042 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.161016    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg"]
Jul 24 04:00:00.161204 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.161160    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747760-b7zw9"
Jul 24 04:00:00.163836 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.163812    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgb7g\""
Jul 24 04:00:00.164335 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.164317    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg"
Jul 24 04:00:00.168446 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.168402    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747760-b7zw9"]
Jul 24 04:00:00.171294 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.171271    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg"]
Jul 24 04:00:00.174935 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.174910    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747760-b7zw9"
Jul 24 04:00:00.183403 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.183382    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg"
Jul 24 04:00:00.338313 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.338200    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747760-b7zw9"]
Jul 24 04:00:00.341010 ip-10-0-140-227 kubenswrapper[2574]: W0724 04:00:00.340980    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod151ac147_fede_4982_950e_a3467524cc98.slice/crio-562f8a1535de62431888cd26620009d58ce481c5af69646b5da91b4eedc85bec WatchSource:0}: Error finding container 562f8a1535de62431888cd26620009d58ce481c5af69646b5da91b4eedc85bec: Status 404 returned error can't find the container with id 562f8a1535de62431888cd26620009d58ce481c5af69646b5da91b4eedc85bec
Jul 24 04:00:00.365862 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.365834    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg"]
Jul 24 04:00:00.368402 ip-10-0-140-227 kubenswrapper[2574]: W0724 04:00:00.368373    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03bde1b7_00e4_41db_9dbe_6f25cd6252c1.slice/crio-cfd459558e184dfff8d9491648e5abe63018b1ba21e9e43394dad93ac0d4fe80 WatchSource:0}: Error finding container cfd459558e184dfff8d9491648e5abe63018b1ba21e9e43394dad93ac0d4fe80: Status 404 returned error can't find the container with id cfd459558e184dfff8d9491648e5abe63018b1ba21e9e43394dad93ac0d4fe80
Jul 24 04:00:00.996973 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.996930    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg" event={"ID":"03bde1b7-00e4-41db-9dbe-6f25cd6252c1","Type":"ContainerStarted","Data":"6bb6a844fecd007bf86d485f52b96992cc27126f921411421a3c97b8b4c91cc0"}
Jul 24 04:00:00.997160 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.996998    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg" event={"ID":"03bde1b7-00e4-41db-9dbe-6f25cd6252c1","Type":"ContainerStarted","Data":"cfd459558e184dfff8d9491648e5abe63018b1ba21e9e43394dad93ac0d4fe80"}
Jul 24 04:00:00.998751 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.998726    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747760-b7zw9" event={"ID":"151ac147-fede-4982-950e-a3467524cc98","Type":"ContainerStarted","Data":"57af133bb34cdeb8cb073a2c257fab34103754eee0e9dfe02b8dd8547b1f2a3e"}
Jul 24 04:00:00.998882 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:00.998766    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747760-b7zw9" event={"ID":"151ac147-fede-4982-950e-a3467524cc98","Type":"ContainerStarted","Data":"562f8a1535de62431888cd26620009d58ce481c5af69646b5da91b4eedc85bec"}
Jul 24 04:00:01.016841 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:01.016794    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg" podStartSLOduration=1.016779201 podStartE2EDuration="1.016779201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:00:01.015868845 +0000 UTC m=+2555.978962133" watchObservedRunningTime="2026-07-24 04:00:01.016779201 +0000 UTC m=+2555.979872466"
Jul 24 04:00:01.031210 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:01.031163    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747760-b7zw9" podStartSLOduration=1.031146295 podStartE2EDuration="1.031146295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:00:01.030249535 +0000 UTC m=+2555.993342824" watchObservedRunningTime="2026-07-24 04:00:01.031146295 +0000 UTC m=+2555.994239582"
Jul 24 04:00:02.004827 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:02.004781    2574 generic.go:356] "Generic (PLEG): container finished" podID="03bde1b7-00e4-41db-9dbe-6f25cd6252c1" containerID="6bb6a844fecd007bf86d485f52b96992cc27126f921411421a3c97b8b4c91cc0" exitCode=0
Jul 24 04:00:02.005276 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:02.004827    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg" event={"ID":"03bde1b7-00e4-41db-9dbe-6f25cd6252c1","Type":"ContainerDied","Data":"6bb6a844fecd007bf86d485f52b96992cc27126f921411421a3c97b8b4c91cc0"}
Jul 24 04:00:02.006548 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:02.006522    2574 generic.go:356] "Generic (PLEG): container finished" podID="151ac147-fede-4982-950e-a3467524cc98" containerID="57af133bb34cdeb8cb073a2c257fab34103754eee0e9dfe02b8dd8547b1f2a3e" exitCode=0
Jul 24 04:00:02.006678 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:02.006601    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747760-b7zw9" event={"ID":"151ac147-fede-4982-950e-a3467524cc98","Type":"ContainerDied","Data":"57af133bb34cdeb8cb073a2c257fab34103754eee0e9dfe02b8dd8547b1f2a3e"}
Jul 24 04:00:03.177187 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:03.177162    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg"
Jul 24 04:00:03.179275 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:03.179256    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747760-b7zw9"
Jul 24 04:00:04.017767 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:04.017736    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747760-b7zw9"
Jul 24 04:00:04.017956 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:04.017768    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747760-b7zw9" event={"ID":"151ac147-fede-4982-950e-a3467524cc98","Type":"ContainerDied","Data":"562f8a1535de62431888cd26620009d58ce481c5af69646b5da91b4eedc85bec"}
Jul 24 04:00:04.017956 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:04.017821    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="562f8a1535de62431888cd26620009d58ce481c5af69646b5da91b4eedc85bec"
Jul 24 04:00:04.019149 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:04.019123    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg" event={"ID":"03bde1b7-00e4-41db-9dbe-6f25cd6252c1","Type":"ContainerDied","Data":"cfd459558e184dfff8d9491648e5abe63018b1ba21e9e43394dad93ac0d4fe80"}
Jul 24 04:00:04.019268 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:04.019156    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg"
Jul 24 04:00:04.019268 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:04.019162    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfd459558e184dfff8d9491648e5abe63018b1ba21e9e43394dad93ac0d4fe80"
Jul 24 04:00:25.849220 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:25.849183    2574 scope.go:117] "RemoveContainer" containerID="8f493c0481c2c1e914e61541e5923b0d6bd1cf8e97ebd3a1653fd179567aa0d7"
Jul 24 04:00:26.829112 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:26.829080    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg"]
Jul 24 04:00:26.832008 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:26.831980    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-b23325-29747760-r5skg"]
Jul 24 04:00:26.862671 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:26.862633    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"]
Jul 24 04:00:26.863118 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:26.862897    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng" podUID="2d4f1957-1ffd-4cac-82dd-b66cc6a3b537" containerName="payload-processing" containerID="cri-o://35fc99d8ce12112e4a48f521bd302563f350bca8b9dcf9352c5caef584eca2ee" gracePeriod=30
Jul 24 04:00:26.869237 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:26.869200    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"]
Jul 24 04:00:26.869582 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:26.869402    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn" podUID="ae8819bf-5272-4ef9-8441-f8901f6bb75a" containerName="payload-pre-processing" containerID="cri-o://59cc589354b7d00d7533a9ad68e0e347e6001d602253d3dc625cf65e85df6d5a" gracePeriod=30
Jul 24 04:00:27.123364 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.123339    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"
Jul 24 04:00:27.143754 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.143726    2574 generic.go:356] "Generic (PLEG): container finished" podID="ae8819bf-5272-4ef9-8441-f8901f6bb75a" containerID="59cc589354b7d00d7533a9ad68e0e347e6001d602253d3dc625cf65e85df6d5a" exitCode=0
Jul 24 04:00:27.143929 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.143811    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn" event={"ID":"ae8819bf-5272-4ef9-8441-f8901f6bb75a","Type":"ContainerDied","Data":"59cc589354b7d00d7533a9ad68e0e347e6001d602253d3dc625cf65e85df6d5a"}
Jul 24 04:00:27.145187 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.145162    2574 generic.go:356] "Generic (PLEG): container finished" podID="2d4f1957-1ffd-4cac-82dd-b66cc6a3b537" containerID="35fc99d8ce12112e4a48f521bd302563f350bca8b9dcf9352c5caef584eca2ee" exitCode=0
Jul 24 04:00:27.145375 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.145326    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng" event={"ID":"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537","Type":"ContainerDied","Data":"35fc99d8ce12112e4a48f521bd302563f350bca8b9dcf9352c5caef584eca2ee"}
Jul 24 04:00:27.145527 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.145283    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"
Jul 24 04:00:27.145527 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.145410    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng" event={"ID":"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537","Type":"ContainerDied","Data":"c723066d0b0824e0aff4a90f0e0ad3641218408a059b96a8af57375c31a759d8"}
Jul 24 04:00:27.145647 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.145572    2574 scope.go:117] "RemoveContainer" containerID="35fc99d8ce12112e4a48f521bd302563f350bca8b9dcf9352c5caef584eca2ee"
Jul 24 04:00:27.158736 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.158682    2574 scope.go:117] "RemoveContainer" containerID="35fc99d8ce12112e4a48f521bd302563f350bca8b9dcf9352c5caef584eca2ee"
Jul 24 04:00:27.159015 ip-10-0-140-227 kubenswrapper[2574]: E0724 04:00:27.158983    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35fc99d8ce12112e4a48f521bd302563f350bca8b9dcf9352c5caef584eca2ee\": container with ID starting with 35fc99d8ce12112e4a48f521bd302563f350bca8b9dcf9352c5caef584eca2ee not found: ID does not exist" containerID="35fc99d8ce12112e4a48f521bd302563f350bca8b9dcf9352c5caef584eca2ee"
Jul 24 04:00:27.159137 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.159023    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35fc99d8ce12112e4a48f521bd302563f350bca8b9dcf9352c5caef584eca2ee"} err="failed to get container status \"35fc99d8ce12112e4a48f521bd302563f350bca8b9dcf9352c5caef584eca2ee\": rpc error: code = NotFound desc = could not find container \"35fc99d8ce12112e4a48f521bd302563f350bca8b9dcf9352c5caef584eca2ee\": container with ID starting with 35fc99d8ce12112e4a48f521bd302563f350bca8b9dcf9352c5caef584eca2ee not found: ID does not exist"
Jul 24 04:00:27.162171 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.162150    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"
Jul 24 04:00:27.220303 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.220269    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrddd\" (UniqueName: \"kubernetes.io/projected/ae8819bf-5272-4ef9-8441-f8901f6bb75a-kube-api-access-rrddd\") pod \"ae8819bf-5272-4ef9-8441-f8901f6bb75a\" (UID: \"ae8819bf-5272-4ef9-8441-f8901f6bb75a\") "
Jul 24 04:00:27.220550 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.220312    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae8819bf-5272-4ef9-8441-f8901f6bb75a-plugins-config-volume\") pod \"ae8819bf-5272-4ef9-8441-f8901f6bb75a\" (UID: \"ae8819bf-5272-4ef9-8441-f8901f6bb75a\") "
Jul 24 04:00:27.220550 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.220328    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4f1957-1ffd-4cac-82dd-b66cc6a3b537-plugins-config-volume\") pod \"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537\" (UID: \"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537\") "
Jul 24 04:00:27.220550 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.220354    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfqtz\" (UniqueName: \"kubernetes.io/projected/2d4f1957-1ffd-4cac-82dd-b66cc6a3b537-kube-api-access-vfqtz\") pod \"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537\" (UID: \"2d4f1957-1ffd-4cac-82dd-b66cc6a3b537\") "
Jul 24 04:00:27.220792 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.220764    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d4f1957-1ffd-4cac-82dd-b66cc6a3b537-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2d4f1957-1ffd-4cac-82dd-b66cc6a3b537" (UID: "2d4f1957-1ffd-4cac-82dd-b66cc6a3b537"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:00:27.220846 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.220782    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae8819bf-5272-4ef9-8441-f8901f6bb75a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ae8819bf-5272-4ef9-8441-f8901f6bb75a" (UID: "ae8819bf-5272-4ef9-8441-f8901f6bb75a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:00:27.222800 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.222776    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae8819bf-5272-4ef9-8441-f8901f6bb75a-kube-api-access-rrddd" (OuterVolumeSpecName: "kube-api-access-rrddd") pod "ae8819bf-5272-4ef9-8441-f8901f6bb75a" (UID: "ae8819bf-5272-4ef9-8441-f8901f6bb75a"). InnerVolumeSpecName "kube-api-access-rrddd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:00:27.222872 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.222827    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d4f1957-1ffd-4cac-82dd-b66cc6a3b537-kube-api-access-vfqtz" (OuterVolumeSpecName: "kube-api-access-vfqtz") pod "2d4f1957-1ffd-4cac-82dd-b66cc6a3b537" (UID: "2d4f1957-1ffd-4cac-82dd-b66cc6a3b537"). InnerVolumeSpecName "kube-api-access-vfqtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:00:27.321919 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.321882    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrddd\" (UniqueName: \"kubernetes.io/projected/ae8819bf-5272-4ef9-8441-f8901f6bb75a-kube-api-access-rrddd\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:00:27.321919 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.321913    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae8819bf-5272-4ef9-8441-f8901f6bb75a-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:00:27.321919 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.321922    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4f1957-1ffd-4cac-82dd-b66cc6a3b537-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:00:27.322163 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.321931    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfqtz\" (UniqueName: \"kubernetes.io/projected/2d4f1957-1ffd-4cac-82dd-b66cc6a3b537-kube-api-access-vfqtz\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:00:27.468055 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.468017    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"]
Jul 24 04:00:27.474743 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.474715    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b23325-77774b9dd4-v4tng"]
Jul 24 04:00:27.573454 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.573397    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03bde1b7-00e4-41db-9dbe-6f25cd6252c1" path="/var/lib/kubelet/pods/03bde1b7-00e4-41db-9dbe-6f25cd6252c1/volumes"
Jul 24 04:00:27.573801 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:27.573787    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d4f1957-1ffd-4cac-82dd-b66cc6a3b537" path="/var/lib/kubelet/pods/2d4f1957-1ffd-4cac-82dd-b66cc6a3b537/volumes"
Jul 24 04:00:28.153628 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:28.153591    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn" event={"ID":"ae8819bf-5272-4ef9-8441-f8901f6bb75a","Type":"ContainerDied","Data":"8c419671047712e7e451fe30420f1f3783e30eda5fcae97c03dcd780860c2c41"}
Jul 24 04:00:28.154113 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:28.153634    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"
Jul 24 04:00:28.154113 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:28.153655    2574 scope.go:117] "RemoveContainer" containerID="59cc589354b7d00d7533a9ad68e0e347e6001d602253d3dc625cf65e85df6d5a"
Jul 24 04:00:28.170440 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:28.170395    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"]
Jul 24 04:00:28.174662 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:28.174636    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b23325-69455ccf8d-gwjzn"]
Jul 24 04:00:29.576809 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:29.576776    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae8819bf-5272-4ef9-8441-f8901f6bb75a" path="/var/lib/kubelet/pods/ae8819bf-5272-4ef9-8441-f8901f6bb75a/volumes"
Jul 24 04:00:42.548448 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.548381    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"]
Jul 24 04:00:42.548865 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.548787    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="151ac147-fede-4982-950e-a3467524cc98" containerName="cleanup"
Jul 24 04:00:42.548865 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.548799    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="151ac147-fede-4982-950e-a3467524cc98" containerName="cleanup"
Jul 24 04:00:42.548865 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.548810    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d4f1957-1ffd-4cac-82dd-b66cc6a3b537" containerName="payload-processing"
Jul 24 04:00:42.548865 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.548816    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d4f1957-1ffd-4cac-82dd-b66cc6a3b537" containerName="payload-processing"
Jul 24 04:00:42.548865 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.548826    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03bde1b7-00e4-41db-9dbe-6f25cd6252c1" containerName="cleanup"
Jul 24 04:00:42.548865 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.548832    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="03bde1b7-00e4-41db-9dbe-6f25cd6252c1" containerName="cleanup"
Jul 24 04:00:42.548865 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.548852    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae8819bf-5272-4ef9-8441-f8901f6bb75a" containerName="payload-pre-processing"
Jul 24 04:00:42.548865 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.548857    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae8819bf-5272-4ef9-8441-f8901f6bb75a" containerName="payload-pre-processing"
Jul 24 04:00:42.549114 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.548911    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="03bde1b7-00e4-41db-9dbe-6f25cd6252c1" containerName="cleanup"
Jul 24 04:00:42.549114 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.548920    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d4f1957-1ffd-4cac-82dd-b66cc6a3b537" containerName="payload-processing"
Jul 24 04:00:42.549114 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.548929    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="151ac147-fede-4982-950e-a3467524cc98" containerName="cleanup"
Jul 24 04:00:42.549114 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.548937    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae8819bf-5272-4ef9-8441-f8901f6bb75a" containerName="payload-pre-processing"
Jul 24 04:00:42.551097 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.551078    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"
Jul 24 04:00:42.553582 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.553561    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-5b56b9\""
Jul 24 04:00:42.563060 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.563033    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"]
Jul 24 04:00:42.604101 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.604065    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"]
Jul 24 04:00:42.606394 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.606376    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"
Jul 24 04:00:42.610469 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.610440    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-5b56b9-dockercfg-kxvpx\""
Jul 24 04:00:42.618341 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.618313    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"]
Jul 24 04:00:42.676000 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.675960    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3e86efe-35d4-4998-8f6e-b83b2224e584-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt\" (UID: \"e3e86efe-35d4-4998-8f6e-b83b2224e584\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"
Jul 24 04:00:42.676212 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.676025    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6t9w\" (UniqueName: \"kubernetes.io/projected/e3e86efe-35d4-4998-8f6e-b83b2224e584-kube-api-access-n6t9w\") pod \"payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt\" (UID: \"e3e86efe-35d4-4998-8f6e-b83b2224e584\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"
Jul 24 04:00:42.676212 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.676110    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6cc3a33-f412-4f6f-820b-920a437ba02a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj\" (UID: \"e6cc3a33-f412-4f6f-820b-920a437ba02a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"
Jul 24 04:00:42.676212 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.676191    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq7mv\" (UniqueName: \"kubernetes.io/projected/e6cc3a33-f412-4f6f-820b-920a437ba02a-kube-api-access-cq7mv\") pod \"payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj\" (UID: \"e6cc3a33-f412-4f6f-820b-920a437ba02a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"
Jul 24 04:00:42.777215 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.777177    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq7mv\" (UniqueName: \"kubernetes.io/projected/e6cc3a33-f412-4f6f-820b-920a437ba02a-kube-api-access-cq7mv\") pod \"payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj\" (UID: \"e6cc3a33-f412-4f6f-820b-920a437ba02a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"
Jul 24 04:00:42.777465 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.777250    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3e86efe-35d4-4998-8f6e-b83b2224e584-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt\" (UID: \"e3e86efe-35d4-4998-8f6e-b83b2224e584\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"
Jul 24 04:00:42.777465 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.777281    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6t9w\" (UniqueName: \"kubernetes.io/projected/e3e86efe-35d4-4998-8f6e-b83b2224e584-kube-api-access-n6t9w\") pod \"payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt\" (UID: \"e3e86efe-35d4-4998-8f6e-b83b2224e584\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"
Jul 24 04:00:42.777465 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.777327    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6cc3a33-f412-4f6f-820b-920a437ba02a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj\" (UID: \"e6cc3a33-f412-4f6f-820b-920a437ba02a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"
Jul 24 04:00:42.778092 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.778065    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3e86efe-35d4-4998-8f6e-b83b2224e584-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt\" (UID: \"e3e86efe-35d4-4998-8f6e-b83b2224e584\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"
Jul 24 04:00:42.778186 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.778093    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6cc3a33-f412-4f6f-820b-920a437ba02a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj\" (UID: \"e6cc3a33-f412-4f6f-820b-920a437ba02a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"
Jul 24 04:00:42.794323 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.794293    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq7mv\" (UniqueName: \"kubernetes.io/projected/e6cc3a33-f412-4f6f-820b-920a437ba02a-kube-api-access-cq7mv\") pod \"payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj\" (UID: \"e6cc3a33-f412-4f6f-820b-920a437ba02a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"
Jul 24 04:00:42.794670 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.794651    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6t9w\" (UniqueName: \"kubernetes.io/projected/e3e86efe-35d4-4998-8f6e-b83b2224e584-kube-api-access-n6t9w\") pod \"payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt\" (UID: \"e3e86efe-35d4-4998-8f6e-b83b2224e584\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"
Jul 24 04:00:42.861649 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.861560    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"
Jul 24 04:00:42.917005 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:42.916971    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"
Jul 24 04:00:43.028224 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:43.028186    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"]
Jul 24 04:00:43.032157 ip-10-0-140-227 kubenswrapper[2574]: W0724 04:00:43.032123    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e86efe_35d4_4998_8f6e_b83b2224e584.slice/crio-ca213cb09b36b55cf476e38d91c0fb1a6c8966248a793543ade9ab31b8bcbb14 WatchSource:0}: Error finding container ca213cb09b36b55cf476e38d91c0fb1a6c8966248a793543ade9ab31b8bcbb14: Status 404 returned error can't find the container with id ca213cb09b36b55cf476e38d91c0fb1a6c8966248a793543ade9ab31b8bcbb14
Jul 24 04:00:43.107967 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:43.107945    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"]
Jul 24 04:00:43.111323 ip-10-0-140-227 kubenswrapper[2574]: W0724 04:00:43.111299    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6cc3a33_f412_4f6f_820b_920a437ba02a.slice/crio-b466bc1eb17753ac2069a43e590b19306afafe3046136c4d328207e1266857a3 WatchSource:0}: Error finding container b466bc1eb17753ac2069a43e590b19306afafe3046136c4d328207e1266857a3: Status 404 returned error can't find the container with id b466bc1eb17753ac2069a43e590b19306afafe3046136c4d328207e1266857a3
Jul 24 04:00:43.235065 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:43.235018    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt" event={"ID":"e3e86efe-35d4-4998-8f6e-b83b2224e584","Type":"ContainerStarted","Data":"fb5c7fbe5b04d288bad4be1b206db855f08f2cbe9a34bc239d2b51221c041b02"}
Jul 24 04:00:43.235230 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:43.235108    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"
Jul 24 04:00:43.235230 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:43.235135    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt" event={"ID":"e3e86efe-35d4-4998-8f6e-b83b2224e584","Type":"ContainerStarted","Data":"ca213cb09b36b55cf476e38d91c0fb1a6c8966248a793543ade9ab31b8bcbb14"}
Jul 24 04:00:43.237119 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:43.237091    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj" event={"ID":"e6cc3a33-f412-4f6f-820b-920a437ba02a","Type":"ContainerStarted","Data":"4bf1532f9ef5b7e63282b239e7221fe875a29ef2ecdd50d1390b554862b776b9"}
Jul 24 04:00:43.237274 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:43.237141    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj" event={"ID":"e6cc3a33-f412-4f6f-820b-920a437ba02a","Type":"ContainerStarted","Data":"b466bc1eb17753ac2069a43e590b19306afafe3046136c4d328207e1266857a3"}
Jul 24 04:00:43.237274 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:43.237196    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"
Jul 24 04:00:43.258018 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:43.257954    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt" podStartSLOduration=1.257934767 podStartE2EDuration="1.257934767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:00:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:00:43.255069114 +0000 UTC m=+2598.218162402" watchObservedRunningTime="2026-07-24 04:00:43.257934767 +0000 UTC m=+2598.221028058"
Jul 24 04:00:43.284053 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:43.284000    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj" podStartSLOduration=1.283983925 podStartE2EDuration="1.283983925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:00:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:00:43.281776118 +0000 UTC m=+2598.244869418" watchObservedRunningTime="2026-07-24 04:00:43.283983925 +0000 UTC m=+2598.247077212"
Jul 24 04:00:50.793340 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:50.793263    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"]
Jul 24 04:00:50.796874 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:50.796854    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"
Jul 24 04:00:50.800080 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:50.800065    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-f701e0\""
Jul 24 04:00:50.800179 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:50.800095    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-f701e0-dockercfg-42nvh\""
Jul 24 04:00:50.818061 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:50.818024    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"]
Jul 24 04:00:50.861123 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:50.861084    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"]
Jul 24 04:00:50.864384 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:50.864367    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"
Jul 24 04:00:50.875934 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:50.875903    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"]
Jul 24 04:00:50.959709 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:50.959665    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02b6f960-d50f-4984-a221-f970a8fdd191-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc\" (UID: \"02b6f960-d50f-4984-a221-f970a8fdd191\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"
Jul 24 04:00:50.959884 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:50.959720    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fsvz\" (UniqueName: \"kubernetes.io/projected/1e4494ec-288f-489e-b910-2a9193bef1c0-kube-api-access-5fsvz\") pod \"payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5\" (UID: \"1e4494ec-288f-489e-b910-2a9193bef1c0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"
Jul 24 04:00:50.959884 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:50.959779    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e4494ec-288f-489e-b910-2a9193bef1c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5\" (UID: \"1e4494ec-288f-489e-b910-2a9193bef1c0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"
Jul 24 04:00:50.959964 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:50.959891    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gczl\" (UniqueName: \"kubernetes.io/projected/02b6f960-d50f-4984-a221-f970a8fdd191-kube-api-access-4gczl\") pod \"payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc\" (UID: \"02b6f960-d50f-4984-a221-f970a8fdd191\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"
Jul 24 04:00:51.061000 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:51.060904    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gczl\" (UniqueName: \"kubernetes.io/projected/02b6f960-d50f-4984-a221-f970a8fdd191-kube-api-access-4gczl\") pod \"payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc\" (UID: \"02b6f960-d50f-4984-a221-f970a8fdd191\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"
Jul 24 04:00:51.061000 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:51.060984    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02b6f960-d50f-4984-a221-f970a8fdd191-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc\" (UID: \"02b6f960-d50f-4984-a221-f970a8fdd191\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"
Jul 24 04:00:51.061248 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:51.061004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fsvz\" (UniqueName: \"kubernetes.io/projected/1e4494ec-288f-489e-b910-2a9193bef1c0-kube-api-access-5fsvz\") pod \"payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5\" (UID: \"1e4494ec-288f-489e-b910-2a9193bef1c0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"
Jul 24 04:00:51.061248 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:51.061042    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e4494ec-288f-489e-b910-2a9193bef1c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5\" (UID: \"1e4494ec-288f-489e-b910-2a9193bef1c0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"
Jul 24 04:00:51.061762 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:51.061737    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e4494ec-288f-489e-b910-2a9193bef1c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5\" (UID: \"1e4494ec-288f-489e-b910-2a9193bef1c0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"
Jul 24 04:00:51.061877 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:51.061793    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02b6f960-d50f-4984-a221-f970a8fdd191-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc\" (UID: \"02b6f960-d50f-4984-a221-f970a8fdd191\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"
Jul 24 04:00:51.070349 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:51.070322    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gczl\" (UniqueName: \"kubernetes.io/projected/02b6f960-d50f-4984-a221-f970a8fdd191-kube-api-access-4gczl\") pod \"payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc\" (UID: \"02b6f960-d50f-4984-a221-f970a8fdd191\") " pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"
Jul 24 04:00:51.070349 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:51.070346    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fsvz\" (UniqueName: \"kubernetes.io/projected/1e4494ec-288f-489e-b910-2a9193bef1c0-kube-api-access-5fsvz\") pod \"payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5\" (UID: \"1e4494ec-288f-489e-b910-2a9193bef1c0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"
Jul 24 04:00:51.107951 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:51.107913    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"
Jul 24 04:00:51.178628 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:51.178284    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"
Jul 24 04:00:51.263845 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:51.263805    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"]
Jul 24 04:00:51.266205 ip-10-0-140-227 kubenswrapper[2574]: W0724 04:00:51.266178    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e4494ec_288f_489e_b910_2a9193bef1c0.slice/crio-105714e279399c42920a29c82ffcfad7aeca536806cb4973a88dbef5d79e1865 WatchSource:0}: Error finding container 105714e279399c42920a29c82ffcfad7aeca536806cb4973a88dbef5d79e1865: Status 404 returned error can't find the container with id 105714e279399c42920a29c82ffcfad7aeca536806cb4973a88dbef5d79e1865
Jul 24 04:00:51.283325 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:51.283265    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5" event={"ID":"1e4494ec-288f-489e-b910-2a9193bef1c0","Type":"ContainerStarted","Data":"105714e279399c42920a29c82ffcfad7aeca536806cb4973a88dbef5d79e1865"}
Jul 24 04:00:51.338373 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:51.338337    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"]
Jul 24 04:00:51.339071 ip-10-0-140-227 kubenswrapper[2574]: W0724 04:00:51.339040    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02b6f960_d50f_4984_a221_f970a8fdd191.slice/crio-729d6f2fc6fcb60f025bc8c5e3edf86abfb586ec3ad19299ed69e0a6ed319b6d WatchSource:0}: Error finding container 729d6f2fc6fcb60f025bc8c5e3edf86abfb586ec3ad19299ed69e0a6ed319b6d: Status 404 returned error can't find the container with id 729d6f2fc6fcb60f025bc8c5e3edf86abfb586ec3ad19299ed69e0a6ed319b6d
Jul 24 04:00:52.290361 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:52.290322    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5" event={"ID":"1e4494ec-288f-489e-b910-2a9193bef1c0","Type":"ContainerStarted","Data":"83893191a5192190af5f5f26bbddb2da3966c5b027a6df72dbb339adedb8baa5"}
Jul 24 04:00:52.290839 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:52.290484    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"
Jul 24 04:00:52.297204 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:52.295812    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc" event={"ID":"02b6f960-d50f-4984-a221-f970a8fdd191","Type":"ContainerStarted","Data":"30321652e15f962007f407520f89717b7fedae974a8ddd9bb6f7e5d91237b992"}
Jul 24 04:00:52.297204 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:52.295864    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc" event={"ID":"02b6f960-d50f-4984-a221-f970a8fdd191","Type":"ContainerStarted","Data":"729d6f2fc6fcb60f025bc8c5e3edf86abfb586ec3ad19299ed69e0a6ed319b6d"}
Jul 24 04:00:52.297204 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:52.296383    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"
Jul 24 04:00:52.309759 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:52.309639    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5" podStartSLOduration=2.309618026 podStartE2EDuration="2.309618026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:00:52.309096508 +0000 UTC m=+2607.272189797" watchObservedRunningTime="2026-07-24 04:00:52.309618026 +0000 UTC m=+2607.272711314"
Jul 24 04:00:52.326052 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:52.326002    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc" podStartSLOduration=2.325984112 podStartE2EDuration="2.325984112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:00:52.325138534 +0000 UTC m=+2607.288231817" watchObservedRunningTime="2026-07-24 04:00:52.325984112 +0000 UTC m=+2607.289077399"
Jul 24 04:00:54.243684 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:54.243634    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj" podUID="e6cc3a33-f412-4f6f-820b-920a437ba02a" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.68:9004: connect: connection refused"
Jul 24 04:00:54.244149 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:00:54.243921    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"
Jul 24 04:01:03.304509 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:01:03.304479    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"
Jul 24 04:01:04.243158 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:01:04.243124    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"
Jul 24 04:01:04.309443 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:01:04.309384    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"
Jul 24 04:02:25.675503 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:02:25.675409    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 04:02:25.681361 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:02:25.681341    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 04:04:25.914183 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:04:25.914148    2574 scope.go:117] "RemoveContainer" containerID="fd45f25e07183fb1f20a7850e5e05cb8ecc2a8793df05c9801c59e6d5c5a5b8e"
Jul 24 04:05:25.936031 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:05:25.935942    2574 scope.go:117] "RemoveContainer" containerID="9c10af393728dbfe0937c964026bf55c939660ba0df15fe04347d8a8133ca257"
Jul 24 04:06:25.958845 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:25.958812    2574 scope.go:117] "RemoveContainer" containerID="6bb6a844fecd007bf86d485f52b96992cc27126f921411421a3c97b8b4c91cc0"
Jul 24 04:06:25.967551 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:25.967526    2574 scope.go:117] "RemoveContainer" containerID="79aa186fa971a9a2532d7d484bea474fda450b193abf9aca345b9798bd65437f"
Jul 24 04:06:49.817971 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:49.817877    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"]
Jul 24 04:06:49.821832 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:49.821812    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"
Jul 24 04:06:49.826520 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:49.826495    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-733dde\""
Jul 24 04:06:49.837127 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:49.837093    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"]
Jul 24 04:06:49.900215 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:49.900174    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7sch\" (UniqueName: \"kubernetes.io/projected/4d90fe6e-842b-4dfb-bddd-94c25099f691-kube-api-access-w7sch\") pod \"payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47\" (UID: \"4d90fe6e-842b-4dfb-bddd-94c25099f691\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"
Jul 24 04:06:49.900399 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:49.900259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d90fe6e-842b-4dfb-bddd-94c25099f691-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47\" (UID: \"4d90fe6e-842b-4dfb-bddd-94c25099f691\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"
Jul 24 04:06:49.944951 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:49.944912    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"]
Jul 24 04:06:49.949333 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:49.949308    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"
Jul 24 04:06:49.953841 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:49.953811    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-733dde-dockercfg-8594s\""
Jul 24 04:06:49.963800 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:49.963773    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"]
Jul 24 04:06:50.000917 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.000871    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d90fe6e-842b-4dfb-bddd-94c25099f691-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47\" (UID: \"4d90fe6e-842b-4dfb-bddd-94c25099f691\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"
Jul 24 04:06:50.001114 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.000966    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee9f28fe-40aa-4bdb-8127-a92dd1a3621b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-733dde-774677678b-86x94\" (UID: \"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"
Jul 24 04:06:50.001114 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.001071    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7sch\" (UniqueName: \"kubernetes.io/projected/4d90fe6e-842b-4dfb-bddd-94c25099f691-kube-api-access-w7sch\") pod \"payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47\" (UID: \"4d90fe6e-842b-4dfb-bddd-94c25099f691\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"
Jul 24 04:06:50.001207 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.001120    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-428dl\" (UniqueName: \"kubernetes.io/projected/ee9f28fe-40aa-4bdb-8127-a92dd1a3621b-kube-api-access-428dl\") pod \"payload-processing-e2e-ipp-a-733dde-774677678b-86x94\" (UID: \"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"
Jul 24 04:06:50.001719 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.001702    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d90fe6e-842b-4dfb-bddd-94c25099f691-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47\" (UID: \"4d90fe6e-842b-4dfb-bddd-94c25099f691\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"
Jul 24 04:06:50.019267 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.019234    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7sch\" (UniqueName: \"kubernetes.io/projected/4d90fe6e-842b-4dfb-bddd-94c25099f691-kube-api-access-w7sch\") pod \"payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47\" (UID: \"4d90fe6e-842b-4dfb-bddd-94c25099f691\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"
Jul 24 04:06:50.102622 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.102521    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee9f28fe-40aa-4bdb-8127-a92dd1a3621b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-733dde-774677678b-86x94\" (UID: \"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"
Jul 24 04:06:50.102622 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.102592    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-428dl\" (UniqueName: \"kubernetes.io/projected/ee9f28fe-40aa-4bdb-8127-a92dd1a3621b-kube-api-access-428dl\") pod \"payload-processing-e2e-ipp-a-733dde-774677678b-86x94\" (UID: \"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"
Jul 24 04:06:50.103268 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.103247    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee9f28fe-40aa-4bdb-8127-a92dd1a3621b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-733dde-774677678b-86x94\" (UID: \"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"
Jul 24 04:06:50.112997 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.112972    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-428dl\" (UniqueName: \"kubernetes.io/projected/ee9f28fe-40aa-4bdb-8127-a92dd1a3621b-kube-api-access-428dl\") pod \"payload-processing-e2e-ipp-a-733dde-774677678b-86x94\" (UID: \"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"
Jul 24 04:06:50.132861 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.132825    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"
Jul 24 04:06:50.260767 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.260730    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"
Jul 24 04:06:50.284134 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.284110    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"]
Jul 24 04:06:50.287020 ip-10-0-140-227 kubenswrapper[2574]: W0724 04:06:50.286979    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d90fe6e_842b_4dfb_bddd_94c25099f691.slice/crio-92dbe64647f437d708b3925890feb54693463c1f5a3082c2cf3df9948fd5e7f8 WatchSource:0}: Error finding container 92dbe64647f437d708b3925890feb54693463c1f5a3082c2cf3df9948fd5e7f8: Status 404 returned error can't find the container with id 92dbe64647f437d708b3925890feb54693463c1f5a3082c2cf3df9948fd5e7f8
Jul 24 04:06:50.418819 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:50.418725    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"]
Jul 24 04:06:50.421065 ip-10-0-140-227 kubenswrapper[2574]: W0724 04:06:50.421040    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee9f28fe_40aa_4bdb_8127_a92dd1a3621b.slice/crio-a9d4bd56932ce7edfc3448802ae2ca43ec7b19a1b326b4dfaf68227aedff42e8 WatchSource:0}: Error finding container a9d4bd56932ce7edfc3448802ae2ca43ec7b19a1b326b4dfaf68227aedff42e8: Status 404 returned error can't find the container with id a9d4bd56932ce7edfc3448802ae2ca43ec7b19a1b326b4dfaf68227aedff42e8
Jul 24 04:06:51.206726 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:51.206692    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94" event={"ID":"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b","Type":"ContainerStarted","Data":"72d0012849a6a8651e7af29c1c9c336d3782eecdeba9e52eb04fff648d1ee295"}
Jul 24 04:06:51.207208 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:51.206749    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94" event={"ID":"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b","Type":"ContainerStarted","Data":"a9d4bd56932ce7edfc3448802ae2ca43ec7b19a1b326b4dfaf68227aedff42e8"}
Jul 24 04:06:51.207208 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:51.206808    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"
Jul 24 04:06:51.208378 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:51.208353    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47" event={"ID":"4d90fe6e-842b-4dfb-bddd-94c25099f691","Type":"ContainerStarted","Data":"c83e5553fc4c405b850f7b5d5d0f0ffad64c5966a7712934672ff36f7573f45c"}
Jul 24 04:06:51.208517 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:51.208400    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47" event={"ID":"4d90fe6e-842b-4dfb-bddd-94c25099f691","Type":"ContainerStarted","Data":"92dbe64647f437d708b3925890feb54693463c1f5a3082c2cf3df9948fd5e7f8"}
Jul 24 04:06:51.208517 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:51.208458    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"
Jul 24 04:06:51.227039 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:51.226962    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94" podStartSLOduration=2.226939864 podStartE2EDuration="2.226939864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:06:51.22600278 +0000 UTC m=+2966.189096066" watchObservedRunningTime="2026-07-24 04:06:51.226939864 +0000 UTC m=+2966.190033153"
Jul 24 04:06:51.244119 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:06:51.244060    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47" podStartSLOduration=2.244034915 podStartE2EDuration="2.244034915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:06:51.241967101 +0000 UTC m=+2966.205060392" watchObservedRunningTime="2026-07-24 04:06:51.244034915 +0000 UTC m=+2966.207128204"
Jul 24 04:07:02.215960 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:02.215926    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"
Jul 24 04:07:02.216519 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:02.216015    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"
Jul 24 04:07:08.758143 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.758102    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"]
Jul 24 04:07:08.762583 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.762558    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"]
Jul 24 04:07:08.762722 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.762686    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"
Jul 24 04:07:08.765931 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.765911    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"
Jul 24 04:07:08.767521 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.767496    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-0cae5e\""
Jul 24 04:07:08.781211 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.781180    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"]
Jul 24 04:07:08.783565 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.783530    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"]
Jul 24 04:07:08.786911 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.786885    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b472f2-ce98-4845-a2a8-780c84624834-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr\" (UID: \"82b472f2-ce98-4845-a2a8-780c84624834\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"
Jul 24 04:07:08.787026 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.786924    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg\" (UID: \"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"
Jul 24 04:07:08.787026 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.786944    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g8k5\" (UniqueName: \"kubernetes.io/projected/7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a-kube-api-access-7g8k5\") pod \"payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg\" (UID: \"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"
Jul 24 04:07:08.787128 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.787106    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw7xn\" (UniqueName: \"kubernetes.io/projected/82b472f2-ce98-4845-a2a8-780c84624834-kube-api-access-bw7xn\") pod \"payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr\" (UID: \"82b472f2-ce98-4845-a2a8-780c84624834\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"
Jul 24 04:07:08.888524 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.888474    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b472f2-ce98-4845-a2a8-780c84624834-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr\" (UID: \"82b472f2-ce98-4845-a2a8-780c84624834\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"
Jul 24 04:07:08.888731 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.888544    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg\" (UID: \"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"
Jul 24 04:07:08.888731 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.888573    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g8k5\" (UniqueName: \"kubernetes.io/projected/7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a-kube-api-access-7g8k5\") pod \"payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg\" (UID: \"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"
Jul 24 04:07:08.888731 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.888716    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw7xn\" (UniqueName: \"kubernetes.io/projected/82b472f2-ce98-4845-a2a8-780c84624834-kube-api-access-bw7xn\") pod \"payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr\" (UID: \"82b472f2-ce98-4845-a2a8-780c84624834\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"
Jul 24 04:07:08.889451 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.889395    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg\" (UID: \"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"
Jul 24 04:07:08.889587 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.889395    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b472f2-ce98-4845-a2a8-780c84624834-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr\" (UID: \"82b472f2-ce98-4845-a2a8-780c84624834\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"
Jul 24 04:07:08.900547 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.900515    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g8k5\" (UniqueName: \"kubernetes.io/projected/7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a-kube-api-access-7g8k5\") pod \"payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg\" (UID: \"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"
Jul 24 04:07:08.900834 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:08.900811    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw7xn\" (UniqueName: \"kubernetes.io/projected/82b472f2-ce98-4845-a2a8-780c84624834-kube-api-access-bw7xn\") pod \"payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr\" (UID: \"82b472f2-ce98-4845-a2a8-780c84624834\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"
Jul 24 04:07:09.074701 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:09.074598    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"
Jul 24 04:07:09.081923 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:09.081895    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"
Jul 24 04:07:09.230491 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:09.230419    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"]
Jul 24 04:07:09.233258 ip-10-0-140-227 kubenswrapper[2574]: W0724 04:07:09.233200    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b0bcf1b_beba_4c92_99b2_6f61b7b28c8a.slice/crio-686c022fb95db8f14cf820bd49353e4a561590d3faf73564b1ccb09437833562 WatchSource:0}: Error finding container 686c022fb95db8f14cf820bd49353e4a561590d3faf73564b1ccb09437833562: Status 404 returned error can't find the container with id 686c022fb95db8f14cf820bd49353e4a561590d3faf73564b1ccb09437833562
Jul 24 04:07:09.260476 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:09.260331    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"]
Jul 24 04:07:09.269759 ip-10-0-140-227 kubenswrapper[2574]: W0724 04:07:09.269727    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b472f2_ce98_4845_a2a8_780c84624834.slice/crio-d6224d1225a19c73343daa72be775307ba328bfa84142b7d3f83aac801c8fd21 WatchSource:0}: Error finding container d6224d1225a19c73343daa72be775307ba328bfa84142b7d3f83aac801c8fd21: Status 404 returned error can't find the container with id d6224d1225a19c73343daa72be775307ba328bfa84142b7d3f83aac801c8fd21
Jul 24 04:07:09.307560 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:09.307529    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg" event={"ID":"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a","Type":"ContainerStarted","Data":"f7eba568200c1dbcff232afebd6bf5ac08caa7c8bb10ed67cf4efd533702949a"}
Jul 24 04:07:09.307716 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:09.307592    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg" event={"ID":"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a","Type":"ContainerStarted","Data":"686c022fb95db8f14cf820bd49353e4a561590d3faf73564b1ccb09437833562"}
Jul 24 04:07:09.307837 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:09.307780    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"
Jul 24 04:07:09.308932 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:09.308907    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr" event={"ID":"82b472f2-ce98-4845-a2a8-780c84624834","Type":"ContainerStarted","Data":"d6224d1225a19c73343daa72be775307ba328bfa84142b7d3f83aac801c8fd21"}
Jul 24 04:07:09.324690 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:09.324646    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg" podStartSLOduration=1.3246297249999999 podStartE2EDuration="1.324629725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:07:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:07:09.323549973 +0000 UTC m=+2984.286643261" watchObservedRunningTime="2026-07-24 04:07:09.324629725 +0000 UTC m=+2984.287723013"
Jul 24 04:07:10.316635 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:10.316598    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr" event={"ID":"82b472f2-ce98-4845-a2a8-780c84624834","Type":"ContainerStarted","Data":"91a3faaa7333d170fb7575ac82959cbd9942eff368a2a154c97a20600b70fa1e"}
Jul 24 04:07:10.317057 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:10.316674    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"
Jul 24 04:07:10.334999 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:10.334936    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr" podStartSLOduration=2.334913866 podStartE2EDuration="2.334913866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:07:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:07:10.33275152 +0000 UTC m=+2985.295844837" watchObservedRunningTime="2026-07-24 04:07:10.334913866 +0000 UTC m=+2985.298007157"
Jul 24 04:07:20.319076 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:20.319042    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"
Jul 24 04:07:21.324201 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:21.324169    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"
Jul 24 04:07:25.707590 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:25.707558    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 04:07:25.714719 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:25.714692    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 04:07:29.433156 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:29.433126    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cz6z2_2ea869f2-beb3-42bc-b932-d0df7d82515d/payload-processing/0.log"
Jul 24 04:07:29.550899 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:07:29.550863    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr_82b472f2-ce98-4845-a2a8-780c84624834/payload-processing/0.log"
Jul 24 04:08:08.172010 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:08.171973    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-733dde-774677678b-86x94_ee9f28fe-40aa-4bdb-8127-a92dd1a3621b/payload-processing/0.log"
Jul 24 04:08:08.297595 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:08.297564    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cz6z2_2ea869f2-beb3-42bc-b932-d0df7d82515d/payload-processing/0.log"
Jul 24 04:08:08.424375 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:08.424274    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr_82b472f2-ce98-4845-a2a8-780c84624834/payload-processing/0.log"
Jul 24 04:08:13.030789 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.030753    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"]
Jul 24 04:08:13.034837 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.034814    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"
Jul 24 04:08:13.037184 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.037165    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-20e639\""
Jul 24 04:08:13.037296 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.037207    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-20e639-dockercfg-btf7t\""
Jul 24 04:08:13.045586 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.045550    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlfz4\" (UniqueName: \"kubernetes.io/projected/a4ca7e52-77f3-434c-a754-39ffb8f59ba3-kube-api-access-tlfz4\") pod \"payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl\" (UID: \"a4ca7e52-77f3-434c-a754-39ffb8f59ba3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"
Jul 24 04:08:13.045733 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.045610    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4ca7e52-77f3-434c-a754-39ffb8f59ba3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl\" (UID: \"a4ca7e52-77f3-434c-a754-39ffb8f59ba3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"
Jul 24 04:08:13.047626 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.047603    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"]
Jul 24 04:08:13.057894 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.057867    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"]
Jul 24 04:08:13.061574 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.061551    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"
Jul 24 04:08:13.071247 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.071216    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"]
Jul 24 04:08:13.146311 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.146272    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlfz4\" (UniqueName: \"kubernetes.io/projected/a4ca7e52-77f3-434c-a754-39ffb8f59ba3-kube-api-access-tlfz4\") pod \"payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl\" (UID: \"a4ca7e52-77f3-434c-a754-39ffb8f59ba3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"
Jul 24 04:08:13.146548 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.146329    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6nt9\" (UniqueName: \"kubernetes.io/projected/9c0aaaf3-5097-454d-ab3c-98ecfb26e17f-kube-api-access-t6nt9\") pod \"payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69\" (UID: \"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"
Jul 24 04:08:13.146548 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.146447    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4ca7e52-77f3-434c-a754-39ffb8f59ba3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl\" (UID: \"a4ca7e52-77f3-434c-a754-39ffb8f59ba3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"
Jul 24 04:08:13.146723 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.146583    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c0aaaf3-5097-454d-ab3c-98ecfb26e17f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69\" (UID: \"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"
Jul 24 04:08:13.147250 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.147228    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4ca7e52-77f3-434c-a754-39ffb8f59ba3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl\" (UID: \"a4ca7e52-77f3-434c-a754-39ffb8f59ba3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"
Jul 24 04:08:13.154634 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.154585    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlfz4\" (UniqueName: \"kubernetes.io/projected/a4ca7e52-77f3-434c-a754-39ffb8f59ba3-kube-api-access-tlfz4\") pod \"payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl\" (UID: \"a4ca7e52-77f3-434c-a754-39ffb8f59ba3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"
Jul 24 04:08:13.247715 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.247677    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c0aaaf3-5097-454d-ab3c-98ecfb26e17f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69\" (UID: \"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"
Jul 24 04:08:13.247896 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.247752    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6nt9\" (UniqueName: \"kubernetes.io/projected/9c0aaaf3-5097-454d-ab3c-98ecfb26e17f-kube-api-access-t6nt9\") pod \"payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69\" (UID: \"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"
Jul 24 04:08:13.248364 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.248346    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c0aaaf3-5097-454d-ab3c-98ecfb26e17f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69\" (UID: \"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"
Jul 24 04:08:13.256682 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.256653    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6nt9\" (UniqueName: \"kubernetes.io/projected/9c0aaaf3-5097-454d-ab3c-98ecfb26e17f-kube-api-access-t6nt9\") pod \"payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69\" (UID: \"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"
Jul 24 04:08:13.347170 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.347069    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"
Jul 24 04:08:13.373499 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.373468    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"
Jul 24 04:08:13.500662 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.500630    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"]
Jul 24 04:08:13.503302 ip-10-0-140-227 kubenswrapper[2574]: W0724 04:08:13.503273    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ca7e52_77f3_434c_a754_39ffb8f59ba3.slice/crio-49ce66e98243ee2866b8869954d42b39874bc416187f8723c16c96edbcea6fc9 WatchSource:0}: Error finding container 49ce66e98243ee2866b8869954d42b39874bc416187f8723c16c96edbcea6fc9: Status 404 returned error can't find the container with id 49ce66e98243ee2866b8869954d42b39874bc416187f8723c16c96edbcea6fc9
Jul 24 04:08:13.651755 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.651712    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl" event={"ID":"a4ca7e52-77f3-434c-a754-39ffb8f59ba3","Type":"ContainerStarted","Data":"d29b88efbfeb3549b07d1fb9f0b408fc985961d022430a9496bbe68f93fa7d91"}
Jul 24 04:08:13.651940 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.651782    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl" event={"ID":"a4ca7e52-77f3-434c-a754-39ffb8f59ba3","Type":"ContainerStarted","Data":"49ce66e98243ee2866b8869954d42b39874bc416187f8723c16c96edbcea6fc9"}
Jul 24 04:08:13.651940 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.651841    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"
Jul 24 04:08:13.667024 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.666977    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl" podStartSLOduration=0.666963094 podStartE2EDuration="666.963094ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:08:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:08:13.665368559 +0000 UTC m=+3048.628461860" watchObservedRunningTime="2026-07-24 04:08:13.666963094 +0000 UTC m=+3048.630056380"
Jul 24 04:08:13.751557 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:13.751526    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"]
Jul 24 04:08:13.754529 ip-10-0-140-227 kubenswrapper[2574]: W0724 04:08:13.754501    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c0aaaf3_5097_454d_ab3c_98ecfb26e17f.slice/crio-6a487ebf2a6a9859f231ebd7b4e85fa02945f7fd56f4abf7f49b3e9b1cdb7ca7 WatchSource:0}: Error finding container 6a487ebf2a6a9859f231ebd7b4e85fa02945f7fd56f4abf7f49b3e9b1cdb7ca7: Status 404 returned error can't find the container with id 6a487ebf2a6a9859f231ebd7b4e85fa02945f7fd56f4abf7f49b3e9b1cdb7ca7
Jul 24 04:08:14.660406 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:14.660372    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69" event={"ID":"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f","Type":"ContainerStarted","Data":"e77c9dd7a13c85ddc3cf6847162b85e8b8187fa18992c75fde4187d5a28e8114"}
Jul 24 04:08:14.660878 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:14.660440    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69" event={"ID":"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f","Type":"ContainerStarted","Data":"6a487ebf2a6a9859f231ebd7b4e85fa02945f7fd56f4abf7f49b3e9b1cdb7ca7"}
Jul 24 04:08:14.660878 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:14.660616    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"
Jul 24 04:08:14.679994 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:14.679942    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69" podStartSLOduration=1.679924984 podStartE2EDuration="1.679924984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 04:08:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:08:14.678394304 +0000 UTC m=+3049.641487593" watchObservedRunningTime="2026-07-24 04:08:14.679924984 +0000 UTC m=+3049.643018271"
Jul 24 04:08:24.663604 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:24.663521    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"
Jul 24 04:08:25.668388 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:25.668351    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"
Jul 24 04:08:38.084654 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.084615    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"]
Jul 24 04:08:38.085050 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.084896    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69" podUID="9c0aaaf3-5097-454d-ab3c-98ecfb26e17f" containerName="payload-processing" containerID="cri-o://e77c9dd7a13c85ddc3cf6847162b85e8b8187fa18992c75fde4187d5a28e8114" gracePeriod=30
Jul 24 04:08:38.091229 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.091201    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"]
Jul 24 04:08:38.091558 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.091401    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl" podUID="a4ca7e52-77f3-434c-a754-39ffb8f59ba3" containerName="payload-pre-processing" containerID="cri-o://d29b88efbfeb3549b07d1fb9f0b408fc985961d022430a9496bbe68f93fa7d91" gracePeriod=30
Jul 24 04:08:38.364879 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.364852    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"
Jul 24 04:08:38.394954 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.394926    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"
Jul 24 04:08:38.510075 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.510035    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c0aaaf3-5097-454d-ab3c-98ecfb26e17f-plugins-config-volume\") pod \"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f\" (UID: \"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f\") "
Jul 24 04:08:38.510075 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.510074    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4ca7e52-77f3-434c-a754-39ffb8f59ba3-plugins-config-volume\") pod \"a4ca7e52-77f3-434c-a754-39ffb8f59ba3\" (UID: \"a4ca7e52-77f3-434c-a754-39ffb8f59ba3\") "
Jul 24 04:08:38.510295 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.510146    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlfz4\" (UniqueName: \"kubernetes.io/projected/a4ca7e52-77f3-434c-a754-39ffb8f59ba3-kube-api-access-tlfz4\") pod \"a4ca7e52-77f3-434c-a754-39ffb8f59ba3\" (UID: \"a4ca7e52-77f3-434c-a754-39ffb8f59ba3\") "
Jul 24 04:08:38.510295 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.510175    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6nt9\" (UniqueName: \"kubernetes.io/projected/9c0aaaf3-5097-454d-ab3c-98ecfb26e17f-kube-api-access-t6nt9\") pod \"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f\" (UID: \"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f\") "
Jul 24 04:08:38.510524 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.510496    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c0aaaf3-5097-454d-ab3c-98ecfb26e17f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9c0aaaf3-5097-454d-ab3c-98ecfb26e17f" (UID: "9c0aaaf3-5097-454d-ab3c-98ecfb26e17f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:08:38.510660 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.510547    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4ca7e52-77f3-434c-a754-39ffb8f59ba3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a4ca7e52-77f3-434c-a754-39ffb8f59ba3" (UID: "a4ca7e52-77f3-434c-a754-39ffb8f59ba3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:08:38.512464 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.512412    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4ca7e52-77f3-434c-a754-39ffb8f59ba3-kube-api-access-tlfz4" (OuterVolumeSpecName: "kube-api-access-tlfz4") pod "a4ca7e52-77f3-434c-a754-39ffb8f59ba3" (UID: "a4ca7e52-77f3-434c-a754-39ffb8f59ba3"). InnerVolumeSpecName "kube-api-access-tlfz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:08:38.512464 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.512421    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c0aaaf3-5097-454d-ab3c-98ecfb26e17f-kube-api-access-t6nt9" (OuterVolumeSpecName: "kube-api-access-t6nt9") pod "9c0aaaf3-5097-454d-ab3c-98ecfb26e17f" (UID: "9c0aaaf3-5097-454d-ab3c-98ecfb26e17f"). InnerVolumeSpecName "kube-api-access-t6nt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:08:38.611734 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.611645    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c0aaaf3-5097-454d-ab3c-98ecfb26e17f-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:08:38.611734 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.611675    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4ca7e52-77f3-434c-a754-39ffb8f59ba3-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:08:38.611734 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.611685    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlfz4\" (UniqueName: \"kubernetes.io/projected/a4ca7e52-77f3-434c-a754-39ffb8f59ba3-kube-api-access-tlfz4\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:08:38.611734 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.611694    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6nt9\" (UniqueName: \"kubernetes.io/projected/9c0aaaf3-5097-454d-ab3c-98ecfb26e17f-kube-api-access-t6nt9\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:08:38.790680 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.790642    2574 generic.go:356] "Generic (PLEG): container finished" podID="a4ca7e52-77f3-434c-a754-39ffb8f59ba3" containerID="d29b88efbfeb3549b07d1fb9f0b408fc985961d022430a9496bbe68f93fa7d91" exitCode=0
Jul 24 04:08:38.790890 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.790694    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"
Jul 24 04:08:38.790890 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.790726    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl" event={"ID":"a4ca7e52-77f3-434c-a754-39ffb8f59ba3","Type":"ContainerDied","Data":"d29b88efbfeb3549b07d1fb9f0b408fc985961d022430a9496bbe68f93fa7d91"}
Jul 24 04:08:38.790890 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.790795    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl" event={"ID":"a4ca7e52-77f3-434c-a754-39ffb8f59ba3","Type":"ContainerDied","Data":"49ce66e98243ee2866b8869954d42b39874bc416187f8723c16c96edbcea6fc9"}
Jul 24 04:08:38.790890 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.790829    2574 scope.go:117] "RemoveContainer" containerID="d29b88efbfeb3549b07d1fb9f0b408fc985961d022430a9496bbe68f93fa7d91"
Jul 24 04:08:38.792188 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.792166    2574 generic.go:356] "Generic (PLEG): container finished" podID="9c0aaaf3-5097-454d-ab3c-98ecfb26e17f" containerID="e77c9dd7a13c85ddc3cf6847162b85e8b8187fa18992c75fde4187d5a28e8114" exitCode=0
Jul 24 04:08:38.792285 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.792220    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"
Jul 24 04:08:38.792356 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.792217    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69" event={"ID":"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f","Type":"ContainerDied","Data":"e77c9dd7a13c85ddc3cf6847162b85e8b8187fa18992c75fde4187d5a28e8114"}
Jul 24 04:08:38.792398 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.792376    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69" event={"ID":"9c0aaaf3-5097-454d-ab3c-98ecfb26e17f","Type":"ContainerDied","Data":"6a487ebf2a6a9859f231ebd7b4e85fa02945f7fd56f4abf7f49b3e9b1cdb7ca7"}
Jul 24 04:08:38.802265 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.802225    2574 scope.go:117] "RemoveContainer" containerID="d29b88efbfeb3549b07d1fb9f0b408fc985961d022430a9496bbe68f93fa7d91"
Jul 24 04:08:38.803084 ip-10-0-140-227 kubenswrapper[2574]: E0724 04:08:38.802697    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d29b88efbfeb3549b07d1fb9f0b408fc985961d022430a9496bbe68f93fa7d91\": container with ID starting with d29b88efbfeb3549b07d1fb9f0b408fc985961d022430a9496bbe68f93fa7d91 not found: ID does not exist" containerID="d29b88efbfeb3549b07d1fb9f0b408fc985961d022430a9496bbe68f93fa7d91"
Jul 24 04:08:38.803084 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.802734    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d29b88efbfeb3549b07d1fb9f0b408fc985961d022430a9496bbe68f93fa7d91"} err="failed to get container status \"d29b88efbfeb3549b07d1fb9f0b408fc985961d022430a9496bbe68f93fa7d91\": rpc error: code = NotFound desc = could not find container \"d29b88efbfeb3549b07d1fb9f0b408fc985961d022430a9496bbe68f93fa7d91\": container with ID starting with d29b88efbfeb3549b07d1fb9f0b408fc985961d022430a9496bbe68f93fa7d91 not found: ID does not exist"
Jul 24 04:08:38.803084 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.802760    2574 scope.go:117] "RemoveContainer" containerID="e77c9dd7a13c85ddc3cf6847162b85e8b8187fa18992c75fde4187d5a28e8114"
Jul 24 04:08:38.813167 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.813145    2574 scope.go:117] "RemoveContainer" containerID="e77c9dd7a13c85ddc3cf6847162b85e8b8187fa18992c75fde4187d5a28e8114"
Jul 24 04:08:38.813559 ip-10-0-140-227 kubenswrapper[2574]: E0724 04:08:38.813529    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e77c9dd7a13c85ddc3cf6847162b85e8b8187fa18992c75fde4187d5a28e8114\": container with ID starting with e77c9dd7a13c85ddc3cf6847162b85e8b8187fa18992c75fde4187d5a28e8114 not found: ID does not exist" containerID="e77c9dd7a13c85ddc3cf6847162b85e8b8187fa18992c75fde4187d5a28e8114"
Jul 24 04:08:38.813712 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.813686    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e77c9dd7a13c85ddc3cf6847162b85e8b8187fa18992c75fde4187d5a28e8114"} err="failed to get container status \"e77c9dd7a13c85ddc3cf6847162b85e8b8187fa18992c75fde4187d5a28e8114\": rpc error: code = NotFound desc = could not find container \"e77c9dd7a13c85ddc3cf6847162b85e8b8187fa18992c75fde4187d5a28e8114\": container with ID starting with e77c9dd7a13c85ddc3cf6847162b85e8b8187fa18992c75fde4187d5a28e8114 not found: ID does not exist"
Jul 24 04:08:38.816590 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.816569    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"]
Jul 24 04:08:38.820115 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.820095    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-20e639-78b456b784-fgdpl"]
Jul 24 04:08:38.830018 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.829997    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"]
Jul 24 04:08:38.833227 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:38.833203    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-20e639-5fccb44b5d-6js69"]
Jul 24 04:08:39.574062 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:39.574030    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c0aaaf3-5097-454d-ab3c-98ecfb26e17f" path="/var/lib/kubelet/pods/9c0aaaf3-5097-454d-ab3c-98ecfb26e17f/volumes"
Jul 24 04:08:39.574500 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:08:39.574378    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4ca7e52-77f3-434c-a754-39ffb8f59ba3" path="/var/lib/kubelet/pods/a4ca7e52-77f3-434c-a754-39ffb8f59ba3/volumes"
Jul 24 04:09:08.326458 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.326385    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"]
Jul 24 04:09:08.327137 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.326656    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94" podUID="ee9f28fe-40aa-4bdb-8127-a92dd1a3621b" containerName="payload-processing" containerID="cri-o://72d0012849a6a8651e7af29c1c9c336d3782eecdeba9e52eb04fff648d1ee295" gracePeriod=30
Jul 24 04:09:08.336622 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.333906    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"]
Jul 24 04:09:08.336622 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.334142    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47" podUID="4d90fe6e-842b-4dfb-bddd-94c25099f691" containerName="payload-pre-processing" containerID="cri-o://c83e5553fc4c405b850f7b5d5d0f0ffad64c5966a7712934672ff36f7573f45c" gracePeriod=30
Jul 24 04:09:08.628283 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.628260    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"
Jul 24 04:09:08.631541 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.631516    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"
Jul 24 04:09:08.713248 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.713205    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee9f28fe-40aa-4bdb-8127-a92dd1a3621b-plugins-config-volume\") pod \"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b\" (UID: \"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b\") "
Jul 24 04:09:08.713467 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.713265    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7sch\" (UniqueName: \"kubernetes.io/projected/4d90fe6e-842b-4dfb-bddd-94c25099f691-kube-api-access-w7sch\") pod \"4d90fe6e-842b-4dfb-bddd-94c25099f691\" (UID: \"4d90fe6e-842b-4dfb-bddd-94c25099f691\") "
Jul 24 04:09:08.713467 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.713381    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-428dl\" (UniqueName: \"kubernetes.io/projected/ee9f28fe-40aa-4bdb-8127-a92dd1a3621b-kube-api-access-428dl\") pod \"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b\" (UID: \"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b\") "
Jul 24 04:09:08.713554 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.713506    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d90fe6e-842b-4dfb-bddd-94c25099f691-plugins-config-volume\") pod \"4d90fe6e-842b-4dfb-bddd-94c25099f691\" (UID: \"4d90fe6e-842b-4dfb-bddd-94c25099f691\") "
Jul 24 04:09:08.713647 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.713611    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee9f28fe-40aa-4bdb-8127-a92dd1a3621b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ee9f28fe-40aa-4bdb-8127-a92dd1a3621b" (UID: "ee9f28fe-40aa-4bdb-8127-a92dd1a3621b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:09:08.713796 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.713777    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d90fe6e-842b-4dfb-bddd-94c25099f691-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4d90fe6e-842b-4dfb-bddd-94c25099f691" (UID: "4d90fe6e-842b-4dfb-bddd-94c25099f691"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:09:08.713909 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.713892    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d90fe6e-842b-4dfb-bddd-94c25099f691-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:09:08.713974 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.713917    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee9f28fe-40aa-4bdb-8127-a92dd1a3621b-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:09:08.715702 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.715681    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee9f28fe-40aa-4bdb-8127-a92dd1a3621b-kube-api-access-428dl" (OuterVolumeSpecName: "kube-api-access-428dl") pod "ee9f28fe-40aa-4bdb-8127-a92dd1a3621b" (UID: "ee9f28fe-40aa-4bdb-8127-a92dd1a3621b"). InnerVolumeSpecName "kube-api-access-428dl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:09:08.715702 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.715689    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d90fe6e-842b-4dfb-bddd-94c25099f691-kube-api-access-w7sch" (OuterVolumeSpecName: "kube-api-access-w7sch") pod "4d90fe6e-842b-4dfb-bddd-94c25099f691" (UID: "4d90fe6e-842b-4dfb-bddd-94c25099f691"). InnerVolumeSpecName "kube-api-access-w7sch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:09:08.814598 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.814555    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7sch\" (UniqueName: \"kubernetes.io/projected/4d90fe6e-842b-4dfb-bddd-94c25099f691-kube-api-access-w7sch\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:09:08.814598 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.814595    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-428dl\" (UniqueName: \"kubernetes.io/projected/ee9f28fe-40aa-4bdb-8127-a92dd1a3621b-kube-api-access-428dl\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:09:08.955209 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.955172    2574 generic.go:356] "Generic (PLEG): container finished" podID="ee9f28fe-40aa-4bdb-8127-a92dd1a3621b" containerID="72d0012849a6a8651e7af29c1c9c336d3782eecdeba9e52eb04fff648d1ee295" exitCode=0
Jul 24 04:09:08.955417 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.955228    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"
Jul 24 04:09:08.955417 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.955252    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94" event={"ID":"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b","Type":"ContainerDied","Data":"72d0012849a6a8651e7af29c1c9c336d3782eecdeba9e52eb04fff648d1ee295"}
Jul 24 04:09:08.955417 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.955332    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94" event={"ID":"ee9f28fe-40aa-4bdb-8127-a92dd1a3621b","Type":"ContainerDied","Data":"a9d4bd56932ce7edfc3448802ae2ca43ec7b19a1b326b4dfaf68227aedff42e8"}
Jul 24 04:09:08.955417 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.955368    2574 scope.go:117] "RemoveContainer" containerID="72d0012849a6a8651e7af29c1c9c336d3782eecdeba9e52eb04fff648d1ee295"
Jul 24 04:09:08.956718 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.956689    2574 generic.go:356] "Generic (PLEG): container finished" podID="4d90fe6e-842b-4dfb-bddd-94c25099f691" containerID="c83e5553fc4c405b850f7b5d5d0f0ffad64c5966a7712934672ff36f7573f45c" exitCode=0
Jul 24 04:09:08.956845 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.956749    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"
Jul 24 04:09:08.956845 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.956782    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47" event={"ID":"4d90fe6e-842b-4dfb-bddd-94c25099f691","Type":"ContainerDied","Data":"c83e5553fc4c405b850f7b5d5d0f0ffad64c5966a7712934672ff36f7573f45c"}
Jul 24 04:09:08.956845 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.956824    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47" event={"ID":"4d90fe6e-842b-4dfb-bddd-94c25099f691","Type":"ContainerDied","Data":"92dbe64647f437d708b3925890feb54693463c1f5a3082c2cf3df9948fd5e7f8"}
Jul 24 04:09:08.966546 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.966524    2574 scope.go:117] "RemoveContainer" containerID="72d0012849a6a8651e7af29c1c9c336d3782eecdeba9e52eb04fff648d1ee295"
Jul 24 04:09:08.966984 ip-10-0-140-227 kubenswrapper[2574]: E0724 04:09:08.966868    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72d0012849a6a8651e7af29c1c9c336d3782eecdeba9e52eb04fff648d1ee295\": container with ID starting with 72d0012849a6a8651e7af29c1c9c336d3782eecdeba9e52eb04fff648d1ee295 not found: ID does not exist" containerID="72d0012849a6a8651e7af29c1c9c336d3782eecdeba9e52eb04fff648d1ee295"
Jul 24 04:09:08.966984 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.966905    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72d0012849a6a8651e7af29c1c9c336d3782eecdeba9e52eb04fff648d1ee295"} err="failed to get container status \"72d0012849a6a8651e7af29c1c9c336d3782eecdeba9e52eb04fff648d1ee295\": rpc error: code = NotFound desc = could not find container \"72d0012849a6a8651e7af29c1c9c336d3782eecdeba9e52eb04fff648d1ee295\": container with ID starting with 72d0012849a6a8651e7af29c1c9c336d3782eecdeba9e52eb04fff648d1ee295 not found: ID does not exist"
Jul 24 04:09:08.966984 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.966930    2574 scope.go:117] "RemoveContainer" containerID="c83e5553fc4c405b850f7b5d5d0f0ffad64c5966a7712934672ff36f7573f45c"
Jul 24 04:09:08.976805 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.976786    2574 scope.go:117] "RemoveContainer" containerID="c83e5553fc4c405b850f7b5d5d0f0ffad64c5966a7712934672ff36f7573f45c"
Jul 24 04:09:08.977135 ip-10-0-140-227 kubenswrapper[2574]: E0724 04:09:08.977115    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c83e5553fc4c405b850f7b5d5d0f0ffad64c5966a7712934672ff36f7573f45c\": container with ID starting with c83e5553fc4c405b850f7b5d5d0f0ffad64c5966a7712934672ff36f7573f45c not found: ID does not exist" containerID="c83e5553fc4c405b850f7b5d5d0f0ffad64c5966a7712934672ff36f7573f45c"
Jul 24 04:09:08.977229 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.977145    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c83e5553fc4c405b850f7b5d5d0f0ffad64c5966a7712934672ff36f7573f45c"} err="failed to get container status \"c83e5553fc4c405b850f7b5d5d0f0ffad64c5966a7712934672ff36f7573f45c\": rpc error: code = NotFound desc = could not find container \"c83e5553fc4c405b850f7b5d5d0f0ffad64c5966a7712934672ff36f7573f45c\": container with ID starting with c83e5553fc4c405b850f7b5d5d0f0ffad64c5966a7712934672ff36f7573f45c not found: ID does not exist"
Jul 24 04:09:08.978014 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.977991    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"]
Jul 24 04:09:08.983696 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.983670    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-733dde-774677678b-86x94"]
Jul 24 04:09:08.993095 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.993070    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"]
Jul 24 04:09:08.998826 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:08.998803    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-733dde-6fff6b497b-kcp47"]
Jul 24 04:09:09.573759 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:09.573722    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d90fe6e-842b-4dfb-bddd-94c25099f691" path="/var/lib/kubelet/pods/4d90fe6e-842b-4dfb-bddd-94c25099f691/volumes"
Jul 24 04:09:09.574188 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:09:09.574090    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee9f28fe-40aa-4bdb-8127-a92dd1a3621b" path="/var/lib/kubelet/pods/ee9f28fe-40aa-4bdb-8127-a92dd1a3621b/volumes"
Jul 24 04:10:12.223294 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.223244    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"]
Jul 24 04:10:12.223949 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.223519    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr" podUID="82b472f2-ce98-4845-a2a8-780c84624834" containerName="payload-processing" containerID="cri-o://91a3faaa7333d170fb7575ac82959cbd9942eff368a2a154c97a20600b70fa1e" gracePeriod=30
Jul 24 04:10:12.230474 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.230416    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"]
Jul 24 04:10:12.231039 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.231001    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg" podUID="7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a" containerName="payload-pre-processing" containerID="cri-o://f7eba568200c1dbcff232afebd6bf5ac08caa7c8bb10ed67cf4efd533702949a" gracePeriod=30
Jul 24 04:10:12.532218 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.532187    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"
Jul 24 04:10:12.535145 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.535123    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"
Jul 24 04:10:12.625617 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.625569    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bw7xn\" (UniqueName: \"kubernetes.io/projected/82b472f2-ce98-4845-a2a8-780c84624834-kube-api-access-bw7xn\") pod \"82b472f2-ce98-4845-a2a8-780c84624834\" (UID: \"82b472f2-ce98-4845-a2a8-780c84624834\") "
Jul 24 04:10:12.625617 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.625628    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b472f2-ce98-4845-a2a8-780c84624834-plugins-config-volume\") pod \"82b472f2-ce98-4845-a2a8-780c84624834\" (UID: \"82b472f2-ce98-4845-a2a8-780c84624834\") "
Jul 24 04:10:12.625876 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.625652    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7g8k5\" (UniqueName: \"kubernetes.io/projected/7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a-kube-api-access-7g8k5\") pod \"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a\" (UID: \"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a\") "
Jul 24 04:10:12.625876 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.625750    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a-plugins-config-volume\") pod \"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a\" (UID: \"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a\") "
Jul 24 04:10:12.626031 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.626004    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82b472f2-ce98-4845-a2a8-780c84624834-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82b472f2-ce98-4845-a2a8-780c84624834" (UID: "82b472f2-ce98-4845-a2a8-780c84624834"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:10:12.626208 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.626181    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a" (UID: "7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:10:12.626296 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.626244    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b472f2-ce98-4845-a2a8-780c84624834-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:10:12.628025 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.628002    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a-kube-api-access-7g8k5" (OuterVolumeSpecName: "kube-api-access-7g8k5") pod "7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a" (UID: "7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a"). InnerVolumeSpecName "kube-api-access-7g8k5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:10:12.628025 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.628010    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82b472f2-ce98-4845-a2a8-780c84624834-kube-api-access-bw7xn" (OuterVolumeSpecName: "kube-api-access-bw7xn") pod "82b472f2-ce98-4845-a2a8-780c84624834" (UID: "82b472f2-ce98-4845-a2a8-780c84624834"). InnerVolumeSpecName "kube-api-access-bw7xn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:10:12.727140 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.727100    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g8k5\" (UniqueName: \"kubernetes.io/projected/7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a-kube-api-access-7g8k5\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:10:12.727140 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.727132    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:10:12.727140 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:12.727142    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw7xn\" (UniqueName: \"kubernetes.io/projected/82b472f2-ce98-4845-a2a8-780c84624834-kube-api-access-bw7xn\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:10:13.296348 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.296314    2574 generic.go:356] "Generic (PLEG): container finished" podID="82b472f2-ce98-4845-a2a8-780c84624834" containerID="91a3faaa7333d170fb7575ac82959cbd9942eff368a2a154c97a20600b70fa1e" exitCode=0
Jul 24 04:10:13.296793 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.296363    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"
Jul 24 04:10:13.296793 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.296398    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr" event={"ID":"82b472f2-ce98-4845-a2a8-780c84624834","Type":"ContainerDied","Data":"91a3faaa7333d170fb7575ac82959cbd9942eff368a2a154c97a20600b70fa1e"}
Jul 24 04:10:13.296793 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.296477    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr" event={"ID":"82b472f2-ce98-4845-a2a8-780c84624834","Type":"ContainerDied","Data":"d6224d1225a19c73343daa72be775307ba328bfa84142b7d3f83aac801c8fd21"}
Jul 24 04:10:13.296793 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.296509    2574 scope.go:117] "RemoveContainer" containerID="91a3faaa7333d170fb7575ac82959cbd9942eff368a2a154c97a20600b70fa1e"
Jul 24 04:10:13.297907 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.297882    2574 generic.go:356] "Generic (PLEG): container finished" podID="7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a" containerID="f7eba568200c1dbcff232afebd6bf5ac08caa7c8bb10ed67cf4efd533702949a" exitCode=0
Jul 24 04:10:13.297973 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.297939    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"
Jul 24 04:10:13.297973 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.297960    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg" event={"ID":"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a","Type":"ContainerDied","Data":"f7eba568200c1dbcff232afebd6bf5ac08caa7c8bb10ed67cf4efd533702949a"}
Jul 24 04:10:13.298066 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.298007    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg" event={"ID":"7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a","Type":"ContainerDied","Data":"686c022fb95db8f14cf820bd49353e4a561590d3faf73564b1ccb09437833562"}
Jul 24 04:10:13.305677 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.305659    2574 scope.go:117] "RemoveContainer" containerID="91a3faaa7333d170fb7575ac82959cbd9942eff368a2a154c97a20600b70fa1e"
Jul 24 04:10:13.306016 ip-10-0-140-227 kubenswrapper[2574]: E0724 04:10:13.305995    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91a3faaa7333d170fb7575ac82959cbd9942eff368a2a154c97a20600b70fa1e\": container with ID starting with 91a3faaa7333d170fb7575ac82959cbd9942eff368a2a154c97a20600b70fa1e not found: ID does not exist" containerID="91a3faaa7333d170fb7575ac82959cbd9942eff368a2a154c97a20600b70fa1e"
Jul 24 04:10:13.306069 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.306025    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91a3faaa7333d170fb7575ac82959cbd9942eff368a2a154c97a20600b70fa1e"} err="failed to get container status \"91a3faaa7333d170fb7575ac82959cbd9942eff368a2a154c97a20600b70fa1e\": rpc error: code = NotFound desc = could not find container \"91a3faaa7333d170fb7575ac82959cbd9942eff368a2a154c97a20600b70fa1e\": container with ID starting with 91a3faaa7333d170fb7575ac82959cbd9942eff368a2a154c97a20600b70fa1e not found: ID does not exist"
Jul 24 04:10:13.306069 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.306042    2574 scope.go:117] "RemoveContainer" containerID="f7eba568200c1dbcff232afebd6bf5ac08caa7c8bb10ed67cf4efd533702949a"
Jul 24 04:10:13.314654 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.314635    2574 scope.go:117] "RemoveContainer" containerID="f7eba568200c1dbcff232afebd6bf5ac08caa7c8bb10ed67cf4efd533702949a"
Jul 24 04:10:13.314956 ip-10-0-140-227 kubenswrapper[2574]: E0724 04:10:13.314935    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7eba568200c1dbcff232afebd6bf5ac08caa7c8bb10ed67cf4efd533702949a\": container with ID starting with f7eba568200c1dbcff232afebd6bf5ac08caa7c8bb10ed67cf4efd533702949a not found: ID does not exist" containerID="f7eba568200c1dbcff232afebd6bf5ac08caa7c8bb10ed67cf4efd533702949a"
Jul 24 04:10:13.315011 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.314965    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7eba568200c1dbcff232afebd6bf5ac08caa7c8bb10ed67cf4efd533702949a"} err="failed to get container status \"f7eba568200c1dbcff232afebd6bf5ac08caa7c8bb10ed67cf4efd533702949a\": rpc error: code = NotFound desc = could not find container \"f7eba568200c1dbcff232afebd6bf5ac08caa7c8bb10ed67cf4efd533702949a\": container with ID starting with f7eba568200c1dbcff232afebd6bf5ac08caa7c8bb10ed67cf4efd533702949a not found: ID does not exist"
Jul 24 04:10:13.322745 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.322717    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"]
Jul 24 04:10:13.326218 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.326193    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-0cae5e-5b844d94d4-kcdzg"]
Jul 24 04:10:13.336016 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.335987    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"]
Jul 24 04:10:13.339679 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.339656    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-0cae5e-85f9fc874-rfqbr"]
Jul 24 04:10:13.574170 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.574091    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a" path="/var/lib/kubelet/pods/7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a/volumes"
Jul 24 04:10:13.574500 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:13.574485    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82b472f2-ce98-4845-a2a8-780c84624834" path="/var/lib/kubelet/pods/82b472f2-ce98-4845-a2a8-780c84624834/volumes"
Jul 24 04:10:33.965168 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:33.965134    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"]
Jul 24 04:10:33.965674 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:33.965406    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt" podUID="e3e86efe-35d4-4998-8f6e-b83b2224e584" containerName="payload-processing" containerID="cri-o://fb5c7fbe5b04d288bad4be1b206db855f08f2cbe9a34bc239d2b51221c041b02" gracePeriod=30
Jul 24 04:10:33.974178 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:33.974143    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"]
Jul 24 04:10:33.974631 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:33.974404    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj" podUID="e6cc3a33-f412-4f6f-820b-920a437ba02a" containerName="payload-pre-processing" containerID="cri-o://4bf1532f9ef5b7e63282b239e7221fe875a29ef2ecdd50d1390b554862b776b9" gracePeriod=30
Jul 24 04:10:34.265624 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.265595    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"
Jul 24 04:10:34.268500 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.268479    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"
Jul 24 04:10:34.341126 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.341095    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3e86efe-35d4-4998-8f6e-b83b2224e584-plugins-config-volume\") pod \"e3e86efe-35d4-4998-8f6e-b83b2224e584\" (UID: \"e3e86efe-35d4-4998-8f6e-b83b2224e584\") "
Jul 24 04:10:34.341274 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.341135    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6t9w\" (UniqueName: \"kubernetes.io/projected/e3e86efe-35d4-4998-8f6e-b83b2224e584-kube-api-access-n6t9w\") pod \"e3e86efe-35d4-4998-8f6e-b83b2224e584\" (UID: \"e3e86efe-35d4-4998-8f6e-b83b2224e584\") "
Jul 24 04:10:34.341274 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.341170    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6cc3a33-f412-4f6f-820b-920a437ba02a-plugins-config-volume\") pod \"e6cc3a33-f412-4f6f-820b-920a437ba02a\" (UID: \"e6cc3a33-f412-4f6f-820b-920a437ba02a\") "
Jul 24 04:10:34.341274 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.341191    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cq7mv\" (UniqueName: \"kubernetes.io/projected/e6cc3a33-f412-4f6f-820b-920a437ba02a-kube-api-access-cq7mv\") pod \"e6cc3a33-f412-4f6f-820b-920a437ba02a\" (UID: \"e6cc3a33-f412-4f6f-820b-920a437ba02a\") "
Jul 24 04:10:34.341606 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.341577    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3e86efe-35d4-4998-8f6e-b83b2224e584-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e3e86efe-35d4-4998-8f6e-b83b2224e584" (UID: "e3e86efe-35d4-4998-8f6e-b83b2224e584"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:10:34.341668 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.341579    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6cc3a33-f412-4f6f-820b-920a437ba02a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6cc3a33-f412-4f6f-820b-920a437ba02a" (UID: "e6cc3a33-f412-4f6f-820b-920a437ba02a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:10:34.343778 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.343710    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6cc3a33-f412-4f6f-820b-920a437ba02a-kube-api-access-cq7mv" (OuterVolumeSpecName: "kube-api-access-cq7mv") pod "e6cc3a33-f412-4f6f-820b-920a437ba02a" (UID: "e6cc3a33-f412-4f6f-820b-920a437ba02a"). InnerVolumeSpecName "kube-api-access-cq7mv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:10:34.343892 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.343818    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3e86efe-35d4-4998-8f6e-b83b2224e584-kube-api-access-n6t9w" (OuterVolumeSpecName: "kube-api-access-n6t9w") pod "e3e86efe-35d4-4998-8f6e-b83b2224e584" (UID: "e3e86efe-35d4-4998-8f6e-b83b2224e584"). InnerVolumeSpecName "kube-api-access-n6t9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:10:34.404789 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.404754    2574 generic.go:356] "Generic (PLEG): container finished" podID="e6cc3a33-f412-4f6f-820b-920a437ba02a" containerID="4bf1532f9ef5b7e63282b239e7221fe875a29ef2ecdd50d1390b554862b776b9" exitCode=0
Jul 24 04:10:34.404957 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.404795    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"
Jul 24 04:10:34.404957 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.404835    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj" event={"ID":"e6cc3a33-f412-4f6f-820b-920a437ba02a","Type":"ContainerDied","Data":"4bf1532f9ef5b7e63282b239e7221fe875a29ef2ecdd50d1390b554862b776b9"}
Jul 24 04:10:34.404957 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.404882    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj" event={"ID":"e6cc3a33-f412-4f6f-820b-920a437ba02a","Type":"ContainerDied","Data":"b466bc1eb17753ac2069a43e590b19306afafe3046136c4d328207e1266857a3"}
Jul 24 04:10:34.404957 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.404908    2574 scope.go:117] "RemoveContainer" containerID="4bf1532f9ef5b7e63282b239e7221fe875a29ef2ecdd50d1390b554862b776b9"
Jul 24 04:10:34.406348 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.406323    2574 generic.go:356] "Generic (PLEG): container finished" podID="e3e86efe-35d4-4998-8f6e-b83b2224e584" containerID="fb5c7fbe5b04d288bad4be1b206db855f08f2cbe9a34bc239d2b51221c041b02" exitCode=0
Jul 24 04:10:34.406510 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.406383    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"
Jul 24 04:10:34.406510 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.406385    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt" event={"ID":"e3e86efe-35d4-4998-8f6e-b83b2224e584","Type":"ContainerDied","Data":"fb5c7fbe5b04d288bad4be1b206db855f08f2cbe9a34bc239d2b51221c041b02"}
Jul 24 04:10:34.406634 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.406568    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt" event={"ID":"e3e86efe-35d4-4998-8f6e-b83b2224e584","Type":"ContainerDied","Data":"ca213cb09b36b55cf476e38d91c0fb1a6c8966248a793543ade9ab31b8bcbb14"}
Jul 24 04:10:34.414846 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.414826    2574 scope.go:117] "RemoveContainer" containerID="4bf1532f9ef5b7e63282b239e7221fe875a29ef2ecdd50d1390b554862b776b9"
Jul 24 04:10:34.415139 ip-10-0-140-227 kubenswrapper[2574]: E0724 04:10:34.415116    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf1532f9ef5b7e63282b239e7221fe875a29ef2ecdd50d1390b554862b776b9\": container with ID starting with 4bf1532f9ef5b7e63282b239e7221fe875a29ef2ecdd50d1390b554862b776b9 not found: ID does not exist" containerID="4bf1532f9ef5b7e63282b239e7221fe875a29ef2ecdd50d1390b554862b776b9"
Jul 24 04:10:34.415221 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.415151    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf1532f9ef5b7e63282b239e7221fe875a29ef2ecdd50d1390b554862b776b9"} err="failed to get container status \"4bf1532f9ef5b7e63282b239e7221fe875a29ef2ecdd50d1390b554862b776b9\": rpc error: code = NotFound desc = could not find container \"4bf1532f9ef5b7e63282b239e7221fe875a29ef2ecdd50d1390b554862b776b9\": container with ID starting with 4bf1532f9ef5b7e63282b239e7221fe875a29ef2ecdd50d1390b554862b776b9 not found: ID does not exist"
Jul 24 04:10:34.415221 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.415175    2574 scope.go:117] "RemoveContainer" containerID="fb5c7fbe5b04d288bad4be1b206db855f08f2cbe9a34bc239d2b51221c041b02"
Jul 24 04:10:34.424293 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.424275    2574 scope.go:117] "RemoveContainer" containerID="fb5c7fbe5b04d288bad4be1b206db855f08f2cbe9a34bc239d2b51221c041b02"
Jul 24 04:10:34.424576 ip-10-0-140-227 kubenswrapper[2574]: E0724 04:10:34.424555    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb5c7fbe5b04d288bad4be1b206db855f08f2cbe9a34bc239d2b51221c041b02\": container with ID starting with fb5c7fbe5b04d288bad4be1b206db855f08f2cbe9a34bc239d2b51221c041b02 not found: ID does not exist" containerID="fb5c7fbe5b04d288bad4be1b206db855f08f2cbe9a34bc239d2b51221c041b02"
Jul 24 04:10:34.424639 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.424586    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb5c7fbe5b04d288bad4be1b206db855f08f2cbe9a34bc239d2b51221c041b02"} err="failed to get container status \"fb5c7fbe5b04d288bad4be1b206db855f08f2cbe9a34bc239d2b51221c041b02\": rpc error: code = NotFound desc = could not find container \"fb5c7fbe5b04d288bad4be1b206db855f08f2cbe9a34bc239d2b51221c041b02\": container with ID starting with fb5c7fbe5b04d288bad4be1b206db855f08f2cbe9a34bc239d2b51221c041b02 not found: ID does not exist"
Jul 24 04:10:34.430129 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.430106    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"]
Jul 24 04:10:34.433445 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.433405    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt"]
Jul 24 04:10:34.442383 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.442360    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3e86efe-35d4-4998-8f6e-b83b2224e584-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:10:34.442383 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.442385    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6t9w\" (UniqueName: \"kubernetes.io/projected/e3e86efe-35d4-4998-8f6e-b83b2224e584-kube-api-access-n6t9w\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:10:34.442559 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.442396    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6cc3a33-f412-4f6f-820b-920a437ba02a-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:10:34.442559 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.442405    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cq7mv\" (UniqueName: \"kubernetes.io/projected/e6cc3a33-f412-4f6f-820b-920a437ba02a-kube-api-access-cq7mv\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:10:34.443706 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.443688    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"]
Jul 24 04:10:34.448414 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:34.448394    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj"]
Jul 24 04:10:35.242964 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:35.242925    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-5b56b9-7c9c89b49d-nxsgt" podUID="e3e86efe-35d4-4998-8f6e-b83b2224e584" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.67:9004: i/o timeout"
Jul 24 04:10:35.243362 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:35.242927    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5b56b9-b56f9fb48-7d4dj" podUID="e6cc3a33-f412-4f6f-820b-920a437ba02a" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.68:9004: i/o timeout"
Jul 24 04:10:35.573819 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:35.573730    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3e86efe-35d4-4998-8f6e-b83b2224e584" path="/var/lib/kubelet/pods/e3e86efe-35d4-4998-8f6e-b83b2224e584/volumes"
Jul 24 04:10:35.574104 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:35.574090    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6cc3a33-f412-4f6f-820b-920a437ba02a" path="/var/lib/kubelet/pods/e6cc3a33-f412-4f6f-820b-920a437ba02a/volumes"
Jul 24 04:10:56.311267 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.311227    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"]
Jul 24 04:10:56.311917 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.311538    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc" podUID="02b6f960-d50f-4984-a221-f970a8fdd191" containerName="payload-processing" containerID="cri-o://30321652e15f962007f407520f89717b7fedae974a8ddd9bb6f7e5d91237b992" gracePeriod=30
Jul 24 04:10:56.319875 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.319845    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"]
Jul 24 04:10:56.320116 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.320089    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5" podUID="1e4494ec-288f-489e-b910-2a9193bef1c0" containerName="payload-pre-processing" containerID="cri-o://83893191a5192190af5f5f26bbddb2da3966c5b027a6df72dbb339adedb8baa5" gracePeriod=30
Jul 24 04:10:56.522577 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.522546    2574 generic.go:356] "Generic (PLEG): container finished" podID="1e4494ec-288f-489e-b910-2a9193bef1c0" containerID="83893191a5192190af5f5f26bbddb2da3966c5b027a6df72dbb339adedb8baa5" exitCode=0
Jul 24 04:10:56.522763 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.522623    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5" event={"ID":"1e4494ec-288f-489e-b910-2a9193bef1c0","Type":"ContainerDied","Data":"83893191a5192190af5f5f26bbddb2da3966c5b027a6df72dbb339adedb8baa5"}
Jul 24 04:10:56.524643 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.524580    2574 generic.go:356] "Generic (PLEG): container finished" podID="02b6f960-d50f-4984-a221-f970a8fdd191" containerID="30321652e15f962007f407520f89717b7fedae974a8ddd9bb6f7e5d91237b992" exitCode=0
Jul 24 04:10:56.524643 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.524624    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc" event={"ID":"02b6f960-d50f-4984-a221-f970a8fdd191","Type":"ContainerDied","Data":"30321652e15f962007f407520f89717b7fedae974a8ddd9bb6f7e5d91237b992"}
Jul 24 04:10:56.588563 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.588536    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"
Jul 24 04:10:56.624278 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.624250    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"
Jul 24 04:10:56.665357 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.665314    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02b6f960-d50f-4984-a221-f970a8fdd191-plugins-config-volume\") pod \"02b6f960-d50f-4984-a221-f970a8fdd191\" (UID: \"02b6f960-d50f-4984-a221-f970a8fdd191\") "
Jul 24 04:10:56.665577 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.665395    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gczl\" (UniqueName: \"kubernetes.io/projected/02b6f960-d50f-4984-a221-f970a8fdd191-kube-api-access-4gczl\") pod \"02b6f960-d50f-4984-a221-f970a8fdd191\" (UID: \"02b6f960-d50f-4984-a221-f970a8fdd191\") "
Jul 24 04:10:56.665577 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.665455    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fsvz\" (UniqueName: \"kubernetes.io/projected/1e4494ec-288f-489e-b910-2a9193bef1c0-kube-api-access-5fsvz\") pod \"1e4494ec-288f-489e-b910-2a9193bef1c0\" (UID: \"1e4494ec-288f-489e-b910-2a9193bef1c0\") "
Jul 24 04:10:56.665577 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.665509    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e4494ec-288f-489e-b910-2a9193bef1c0-plugins-config-volume\") pod \"1e4494ec-288f-489e-b910-2a9193bef1c0\" (UID: \"1e4494ec-288f-489e-b910-2a9193bef1c0\") "
Jul 24 04:10:56.665762 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.665704    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02b6f960-d50f-4984-a221-f970a8fdd191-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "02b6f960-d50f-4984-a221-f970a8fdd191" (UID: "02b6f960-d50f-4984-a221-f970a8fdd191"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:10:56.665909 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.665885    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02b6f960-d50f-4984-a221-f970a8fdd191-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:10:56.666002 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.665984    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e4494ec-288f-489e-b910-2a9193bef1c0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1e4494ec-288f-489e-b910-2a9193bef1c0" (UID: "1e4494ec-288f-489e-b910-2a9193bef1c0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 04:10:56.667936 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.667908    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e4494ec-288f-489e-b910-2a9193bef1c0-kube-api-access-5fsvz" (OuterVolumeSpecName: "kube-api-access-5fsvz") pod "1e4494ec-288f-489e-b910-2a9193bef1c0" (UID: "1e4494ec-288f-489e-b910-2a9193bef1c0"). InnerVolumeSpecName "kube-api-access-5fsvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:10:56.668033 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.667948    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02b6f960-d50f-4984-a221-f970a8fdd191-kube-api-access-4gczl" (OuterVolumeSpecName: "kube-api-access-4gczl") pod "02b6f960-d50f-4984-a221-f970a8fdd191" (UID: "02b6f960-d50f-4984-a221-f970a8fdd191"). InnerVolumeSpecName "kube-api-access-4gczl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 04:10:56.767180 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.767148    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gczl\" (UniqueName: \"kubernetes.io/projected/02b6f960-d50f-4984-a221-f970a8fdd191-kube-api-access-4gczl\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:10:56.767180 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.767177    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fsvz\" (UniqueName: \"kubernetes.io/projected/1e4494ec-288f-489e-b910-2a9193bef1c0-kube-api-access-5fsvz\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:10:56.767404 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:56.767190    2574 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e4494ec-288f-489e-b910-2a9193bef1c0-plugins-config-volume\") on node \"ip-10-0-140-227.ec2.internal\" DevicePath \"\""
Jul 24 04:10:57.530810 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:57.530777    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"
Jul 24 04:10:57.531279 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:57.530779    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5" event={"ID":"1e4494ec-288f-489e-b910-2a9193bef1c0","Type":"ContainerDied","Data":"105714e279399c42920a29c82ffcfad7aeca536806cb4973a88dbef5d79e1865"}
Jul 24 04:10:57.531279 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:57.530938    2574 scope.go:117] "RemoveContainer" containerID="83893191a5192190af5f5f26bbddb2da3966c5b027a6df72dbb339adedb8baa5"
Jul 24 04:10:57.539692 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:57.539587    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc" event={"ID":"02b6f960-d50f-4984-a221-f970a8fdd191","Type":"ContainerDied","Data":"729d6f2fc6fcb60f025bc8c5e3edf86abfb586ec3ad19299ed69e0a6ed319b6d"}
Jul 24 04:10:57.539692 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:57.539687    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"
Jul 24 04:10:57.549195 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:57.549122    2574 scope.go:117] "RemoveContainer" containerID="30321652e15f962007f407520f89717b7fedae974a8ddd9bb6f7e5d91237b992"
Jul 24 04:10:57.557675 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:57.557649    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"]
Jul 24 04:10:57.559733 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:57.559705    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-f701e0-7c98f459d4-xvld5"]
Jul 24 04:10:57.575029 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:57.574994    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e4494ec-288f-489e-b910-2a9193bef1c0" path="/var/lib/kubelet/pods/1e4494ec-288f-489e-b910-2a9193bef1c0/volumes"
Jul 24 04:10:57.575377 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:57.575359    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"]
Jul 24 04:10:57.575447 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:57.575391    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-f701e0-7cc956c954-4pmwc"]
Jul 24 04:10:59.572632 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:10:59.572599    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02b6f960-d50f-4984-a221-f970a8fdd191" path="/var/lib/kubelet/pods/02b6f960-d50f-4984-a221-f970a8fdd191/volumes"
Jul 24 04:11:14.986154 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:14.986123    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f4587fcc-6zg4n_68a31406-ae8d-4a3a-b9b4-3740e31aecee/copy-manifests/0.log"
Jul 24 04:11:14.992958 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:14.992934    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f4587fcc-6zg4n_68a31406-ae8d-4a3a-b9b4-3740e31aecee/manager/0.log"
Jul 24 04:11:15.729938 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:15.729910    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747760-b7zw9_151ac147-fede-4982-950e-a3467524cc98/cleanup/0.log"
Jul 24 04:11:17.853440 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:17.853391    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6268t_ef1db0c8-bc87-413a-b402-115057ea1215/manager/0.log"
Jul 24 04:11:18.615260 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:18.615227    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-56l8s_4aedf099-d724-413b-a2de-d960cc12170e/discovery/0.log"
Jul 24 04:11:19.224746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:19.224714    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cz6z2_2ea869f2-beb3-42bc-b932-d0df7d82515d/payload-processing/0.log"
Jul 24 04:11:19.339997 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:19.339966    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-799b44d47c-5c54q_09f9b961-e743-4da0-be0d-80a58f646b62/router/0.log"
Jul 24 04:11:19.819739 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:19.819707    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-9xc96_da8fdad3-4aea-4fd4-8b38-04f1bc0215f1/main/0.log"
Jul 24 04:11:20.052943 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:20.052915    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fpf4dz_c69e11bf-47be-45ef-aec3-e030916a8c52/main/0.log"
Jul 24 04:11:20.165786 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:20.165762    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-zg2zz_4019c381-660b-4515-a6f8-1db607a0ac82/main/0.log"
Jul 24 04:11:28.508328 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:28.508296    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6vl7h_3d93655f-1e04-43cc-ac4b-1cd6fb0a9a1d/global-pull-secret-syncer/0.log"
Jul 24 04:11:28.628052 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:28.628019    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nxrsc_5618041c-75a6-4c3b-a584-519fad1a4b20/konnectivity-agent/0.log"
Jul 24 04:11:28.738957 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:28.738922    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-227.ec2.internal_cb9eba28ae5571b83e98bdb2f8bef7f3/haproxy/0.log"
Jul 24 04:11:34.536170 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:34.536126    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-6268t_ef1db0c8-bc87-413a-b402-115057ea1215/manager/0.log"
Jul 24 04:11:36.478699 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:36.478662    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-p5ghk_136cde74-b9d8-4c3c-9bca-74b7628db11b/cluster-monitoring-operator/0.log"
Jul 24 04:11:36.505722 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:36.505691    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-v9d9r_654d60b6-080a-4e5f-8a60-544e5986bf00/kube-state-metrics/0.log"
Jul 24 04:11:36.529774 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:36.529747    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-v9d9r_654d60b6-080a-4e5f-8a60-544e5986bf00/kube-rbac-proxy-main/0.log"
Jul 24 04:11:36.552185 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:36.552151    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-v9d9r_654d60b6-080a-4e5f-8a60-544e5986bf00/kube-rbac-proxy-self/0.log"
Jul 24 04:11:36.581534 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:36.581506    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5b4fccfb64-hf8zj_7fb24886-365d-4e28-b560-6fb915320fab/metrics-server/0.log"
Jul 24 04:11:36.610367 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:36.610299    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-cdtzk_83b1ba72-f4aa-4d98-9382-65f711fbab75/monitoring-plugin/0.log"
Jul 24 04:11:36.641202 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:36.641171    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2vsf6_737a1d45-07a7-41b6-9e00-28c489b71173/node-exporter/0.log"
Jul 24 04:11:36.665012 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:36.664991    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2vsf6_737a1d45-07a7-41b6-9e00-28c489b71173/kube-rbac-proxy/0.log"
Jul 24 04:11:36.689569 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:36.689542    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2vsf6_737a1d45-07a7-41b6-9e00-28c489b71173/init-textfile/0.log"
Jul 24 04:11:36.891886 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:36.891856    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-q7nq8_b2c90cb8-5136-428b-920b-0db0144e03ba/kube-rbac-proxy-main/0.log"
Jul 24 04:11:36.916168 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:36.916145    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-q7nq8_b2c90cb8-5136-428b-920b-0db0144e03ba/kube-rbac-proxy-self/0.log"
Jul 24 04:11:36.938719 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:36.938688    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-q7nq8_b2c90cb8-5136-428b-920b-0db0144e03ba/openshift-state-metrics/0.log"
Jul 24 04:11:36.980057 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:36.980025    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0d695c4-fc68-4e09-b78e-82fc7b59abb9/prometheus/0.log"
Jul 24 04:11:37.002443 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:37.002410    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0d695c4-fc68-4e09-b78e-82fc7b59abb9/config-reloader/0.log"
Jul 24 04:11:37.023964 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:37.023936    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0d695c4-fc68-4e09-b78e-82fc7b59abb9/thanos-sidecar/0.log"
Jul 24 04:11:37.050997 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:37.050972    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0d695c4-fc68-4e09-b78e-82fc7b59abb9/kube-rbac-proxy-web/0.log"
Jul 24 04:11:37.076724 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:37.076695    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0d695c4-fc68-4e09-b78e-82fc7b59abb9/kube-rbac-proxy/0.log"
Jul 24 04:11:37.107884 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:37.107859    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0d695c4-fc68-4e09-b78e-82fc7b59abb9/kube-rbac-proxy-thanos/0.log"
Jul 24 04:11:37.130722 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:37.130702    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0d695c4-fc68-4e09-b78e-82fc7b59abb9/init-config-reloader/0.log"
Jul 24 04:11:37.169815 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:37.169714    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-465j4_3acf96c6-8064-4d64-b8c5-c479d5ecc856/prometheus-operator/0.log"
Jul 24 04:11:37.186360 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:37.186330    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-465j4_3acf96c6-8064-4d64-b8c5-c479d5ecc856/kube-rbac-proxy/0.log"
Jul 24 04:11:38.670910 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.670875    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"]
Jul 24 04:11:38.671465 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671420    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e4494ec-288f-489e-b910-2a9193bef1c0" containerName="payload-pre-processing"
Jul 24 04:11:38.671518 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671467    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e4494ec-288f-489e-b910-2a9193bef1c0" containerName="payload-pre-processing"
Jul 24 04:11:38.671518 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671477    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6cc3a33-f412-4f6f-820b-920a437ba02a" containerName="payload-pre-processing"
Jul 24 04:11:38.671518 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671483    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6cc3a33-f412-4f6f-820b-920a437ba02a" containerName="payload-pre-processing"
Jul 24 04:11:38.671518 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671493    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c0aaaf3-5097-454d-ab3c-98ecfb26e17f" containerName="payload-processing"
Jul 24 04:11:38.671518 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671499    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c0aaaf3-5097-454d-ab3c-98ecfb26e17f" containerName="payload-processing"
Jul 24 04:11:38.671518 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671508    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9f28fe-40aa-4bdb-8127-a92dd1a3621b" containerName="payload-processing"
Jul 24 04:11:38.671518 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671514    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9f28fe-40aa-4bdb-8127-a92dd1a3621b" containerName="payload-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671524    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d90fe6e-842b-4dfb-bddd-94c25099f691" containerName="payload-pre-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671529    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d90fe6e-842b-4dfb-bddd-94c25099f691" containerName="payload-pre-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671537    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82b472f2-ce98-4845-a2a8-780c84624834" containerName="payload-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671542    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="82b472f2-ce98-4845-a2a8-780c84624834" containerName="payload-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671548    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4ca7e52-77f3-434c-a754-39ffb8f59ba3" containerName="payload-pre-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671553    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4ca7e52-77f3-434c-a754-39ffb8f59ba3" containerName="payload-pre-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671560    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02b6f960-d50f-4984-a221-f970a8fdd191" containerName="payload-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671565    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="02b6f960-d50f-4984-a221-f970a8fdd191" containerName="payload-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671571    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a" containerName="payload-pre-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671576    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a" containerName="payload-pre-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671585    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3e86efe-35d4-4998-8f6e-b83b2224e584" containerName="payload-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671590    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3e86efe-35d4-4998-8f6e-b83b2224e584" containerName="payload-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671647    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d90fe6e-842b-4dfb-bddd-94c25099f691" containerName="payload-pre-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671656    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="02b6f960-d50f-4984-a221-f970a8fdd191" containerName="payload-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671663    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="82b472f2-ce98-4845-a2a8-780c84624834" containerName="payload-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671670    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee9f28fe-40aa-4bdb-8127-a92dd1a3621b" containerName="payload-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671677    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6cc3a33-f412-4f6f-820b-920a437ba02a" containerName="payload-pre-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671683    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3e86efe-35d4-4998-8f6e-b83b2224e584" containerName="payload-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671688    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e4494ec-288f-489e-b910-2a9193bef1c0" containerName="payload-pre-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671694    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4ca7e52-77f3-434c-a754-39ffb8f59ba3" containerName="payload-pre-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671699    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b0bcf1b-beba-4c92-99b2-6f61b7b28c8a" containerName="payload-pre-processing"
Jul 24 04:11:38.671746 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.671705    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c0aaaf3-5097-454d-ab3c-98ecfb26e17f" containerName="payload-processing"
Jul 24 04:11:38.674919 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.674897    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.677026 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.677001    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tsxt7\"/\"kube-root-ca.crt\""
Jul 24 04:11:38.677775 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.677750    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tsxt7\"/\"openshift-service-ca.crt\""
Jul 24 04:11:38.677886 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.677791    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tsxt7\"/\"default-dockercfg-jw6nv\""
Jul 24 04:11:38.681162 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.681126    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"]
Jul 24 04:11:38.771858 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.771817    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qnsr\" (UniqueName: \"kubernetes.io/projected/1814d0bc-a2b4-4425-b348-49e6694c526a-kube-api-access-2qnsr\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.771858 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.771865    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1814d0bc-a2b4-4425-b348-49e6694c526a-podres\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.772083 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.771939    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1814d0bc-a2b4-4425-b348-49e6694c526a-lib-modules\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.772083 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.771977    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1814d0bc-a2b4-4425-b348-49e6694c526a-proc\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.772083 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.772056    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1814d0bc-a2b4-4425-b348-49e6694c526a-sys\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.873561 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.873518    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1814d0bc-a2b4-4425-b348-49e6694c526a-sys\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.873757 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.873585    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qnsr\" (UniqueName: \"kubernetes.io/projected/1814d0bc-a2b4-4425-b348-49e6694c526a-kube-api-access-2qnsr\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.873757 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.873621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1814d0bc-a2b4-4425-b348-49e6694c526a-podres\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.873757 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.873653    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1814d0bc-a2b4-4425-b348-49e6694c526a-lib-modules\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.873757 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.873660    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1814d0bc-a2b4-4425-b348-49e6694c526a-sys\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.873757 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.873675    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1814d0bc-a2b4-4425-b348-49e6694c526a-proc\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.873956 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.873769    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1814d0bc-a2b4-4425-b348-49e6694c526a-proc\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.873956 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.873793    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1814d0bc-a2b4-4425-b348-49e6694c526a-lib-modules\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.873956 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.873794    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1814d0bc-a2b4-4425-b348-49e6694c526a-podres\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.883557 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.883528    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qnsr\" (UniqueName: \"kubernetes.io/projected/1814d0bc-a2b4-4425-b348-49e6694c526a-kube-api-access-2qnsr\") pod \"perf-node-gather-daemonset-wr49j\" (UID: \"1814d0bc-a2b4-4425-b348-49e6694c526a\") " pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:38.986507 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:38.986392    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:39.121693 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:39.121641    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"]
Jul 24 04:11:39.124289 ip-10-0-140-227 kubenswrapper[2574]: W0724 04:11:39.124257    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1814d0bc_a2b4_4425_b348_49e6694c526a.slice/crio-5c82221b1620f97616aa95277f193607e396330afae4baf6dc112b14ed2d3d9f WatchSource:0}: Error finding container 5c82221b1620f97616aa95277f193607e396330afae4baf6dc112b14ed2d3d9f: Status 404 returned error can't find the container with id 5c82221b1620f97616aa95277f193607e396330afae4baf6dc112b14ed2d3d9f
Jul 24 04:11:39.281973 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:39.281940    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/1.log"
Jul 24 04:11:39.286041 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:39.286020    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bjpkp_69510a47-b326-4920-acb8-1baa907c926d/console-operator/2.log"
Jul 24 04:11:39.751724 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:39.751685    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j" event={"ID":"1814d0bc-a2b4-4425-b348-49e6694c526a","Type":"ContainerStarted","Data":"c7858219ea1f9dc73adf558d469b0ab1432c53767192f4b2cde6514c4b6cf418"}
Jul 24 04:11:39.752131 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:39.751749    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:39.752131 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:39.751764    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j" event={"ID":"1814d0bc-a2b4-4425-b348-49e6694c526a","Type":"ContainerStarted","Data":"5c82221b1620f97616aa95277f193607e396330afae4baf6dc112b14ed2d3d9f"}
Jul 24 04:11:39.769766 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:39.769713    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j" podStartSLOduration=1.769692503 podStartE2EDuration="1.769692503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 04:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 04:11:39.767514591 +0000 UTC m=+3254.730607880" watchObservedRunningTime="2026-07-24 04:11:39.769692503 +0000 UTC m=+3254.732785831"
Jul 24 04:11:39.830844 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:39.830817    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-48xzj_985df3af-4c44-462b-90e2-46bbe9a6b181/download-server/0.log"
Jul 24 04:11:40.411284 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:40.411246    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-xbwck_3af1a82c-c889-4da3-95fa-6812a25bcea3/volume-data-source-validator/0.log"
Jul 24 04:11:41.560696 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:41.560675    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tb9dz_3d10c276-6ba4-44b2-9895-1bb803660d4e/dns/0.log"
Jul 24 04:11:41.583405 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:41.583379    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tb9dz_3d10c276-6ba4-44b2-9895-1bb803660d4e/kube-rbac-proxy/0.log"
Jul 24 04:11:41.630129 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:41.630102    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s7jjg_559deb09-e606-40bb-ba17-b957dca7b41a/dns-node-resolver/0.log"
Jul 24 04:11:42.227469 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:42.227421    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5j4wp_4b97304d-62fa-4447-81d8-525c8b6670b3/node-ca/0.log"
Jul 24 04:11:43.404573 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:43.404540    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-56l8s_4aedf099-d724-413b-a2de-d960cc12170e/discovery/0.log"
Jul 24 04:11:43.570641 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:43.570611    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cz6z2_2ea869f2-beb3-42bc-b932-d0df7d82515d/payload-processing/0.log"
Jul 24 04:11:43.598004 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:43.597976    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-799b44d47c-5c54q_09f9b961-e743-4da0-be0d-80a58f646b62/router/0.log"
Jul 24 04:11:44.439719 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:44.439688    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-757zg_fe775e9b-fb53-4c76-b95e-a43de1163af0/serve-healthcheck-canary/0.log"
Jul 24 04:11:45.091633 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:45.091607    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x95zd_bef52dbb-50ae-4945-8232-4da340d6439d/kube-rbac-proxy/0.log"
Jul 24 04:11:45.113800 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:45.113775    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x95zd_bef52dbb-50ae-4945-8232-4da340d6439d/exporter/0.log"
Jul 24 04:11:45.136303 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:45.136278    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x95zd_bef52dbb-50ae-4945-8232-4da340d6439d/extractor/0.log"
Jul 24 04:11:45.766807 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:45.766773    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tsxt7/perf-node-gather-daemonset-wr49j"
Jul 24 04:11:47.972537 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:47.972453    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f4587fcc-6zg4n_68a31406-ae8d-4a3a-b9b4-3740e31aecee/manager/0.log"
Jul 24 04:11:47.998408 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:47.998380    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f4587fcc-6zg4n_68a31406-ae8d-4a3a-b9b4-3740e31aecee/copy-manifests/0.log"
Jul 24 04:11:49.383775 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:49.383747    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-54698f7c78-pjtd8_8659a9e5-cf20-493c-9868-e123a8cba9b1/manager/0.log"
Jul 24 04:11:49.700638 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:49.700599    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-56l8s_4aedf099-d724-413b-a2de-d960cc12170e/discovery/0.log"
Jul 24 04:11:49.818920 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:49.818892    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cz6z2_2ea869f2-beb3-42bc-b932-d0df7d82515d/payload-processing/0.log"
Jul 24 04:11:49.835175 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:49.835147    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-799b44d47c-5c54q_09f9b961-e743-4da0-be0d-80a58f646b62/router/0.log"
Jul 24 04:11:51.414651 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:51.414615    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-56l8s_4aedf099-d724-413b-a2de-d960cc12170e/discovery/0.log"
Jul 24 04:11:51.548568 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:51.548540    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cz6z2_2ea869f2-beb3-42bc-b932-d0df7d82515d/payload-processing/0.log"
Jul 24 04:11:51.571975 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:51.571949    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-799b44d47c-5c54q_09f9b961-e743-4da0-be0d-80a58f646b62/router/0.log"
Jul 24 04:11:55.016554 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:55.016520    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp_bc7cb98c-bd37-44bf-80f0-1cd615e8a36f/kube-storage-version-migrator-operator/1.log"
Jul 24 04:11:55.017417 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:55.017397    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-lnjzp_bc7cb98c-bd37-44bf-80f0-1cd615e8a36f/kube-storage-version-migrator-operator/0.log"
Jul 24 04:11:56.023264 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:56.023237    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wrj9_0d6301d1-c237-48f6-b5af-921a4a4540f1/kube-multus-additional-cni-plugins/0.log"
Jul 24 04:11:56.048034 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:56.048008    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wrj9_0d6301d1-c237-48f6-b5af-921a4a4540f1/egress-router-binary-copy/0.log"
Jul 24 04:11:56.069979 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:56.069955    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wrj9_0d6301d1-c237-48f6-b5af-921a4a4540f1/cni-plugins/0.log"
Jul 24 04:11:56.092344 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:56.092316    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wrj9_0d6301d1-c237-48f6-b5af-921a4a4540f1/bond-cni-plugin/0.log"
Jul 24 04:11:56.115739 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:56.115708    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wrj9_0d6301d1-c237-48f6-b5af-921a4a4540f1/routeoverride-cni/0.log"
Jul 24 04:11:56.138961 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:56.138932    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wrj9_0d6301d1-c237-48f6-b5af-921a4a4540f1/whereabouts-cni-bincopy/0.log"
Jul 24 04:11:56.161051 ip-10-0-140-227 kubenswrapper[2574]: I0724 04:11:56.161025    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wrj9_0d6301d1-c237-48f6-b5af-921a4a4540f1/whereabouts-cni/0.log"