Sep 14 07:47:59.260956 ip-10-0-143-250 systemd[1]: Starting Kubernetes Kubelet...
Sep 14 07:47:59.650460 ip-10-0-143-250 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:47:59.650460 ip-10-0-143-250 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 14 07:47:59.650460 ip-10-0-143-250 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:47:59.650460 ip-10-0-143-250 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 14 07:47:59.650460 ip-10-0-143-250 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:47:59.652561 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.652471    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 14 07:47:59.656787 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656772    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:47:59.656787 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656787    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656791    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656794    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656797    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656800    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656803    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656806    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656814    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656817    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656820    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656822    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656825    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656828    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656830    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656833    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656836    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656838    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656841    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656844    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656847    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:47:59.656883 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656849    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656852    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656854    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656857    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656860    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656862    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656865    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656868    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656870    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656873    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656875    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656879    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656883    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656886    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656889    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656892    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656896    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656899    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656903    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:47:59.657355 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656906    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656908    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656911    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656914    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656916    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656919    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656921    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656924    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656927    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656929    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656932    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656934    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656937    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656939    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656942    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656945    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656948    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656951    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656955    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656957    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:47:59.657857 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656960    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656963    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656965    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656968    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656971    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656974    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656976    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656979    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656982    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656985    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656987    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656991    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656994    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656996    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.656999    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657001    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657004    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657007    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657010    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657012    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:47:59.658344 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657015    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657019    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657022    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657025    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657027    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657032    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657035    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657039    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657042    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657045    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657048    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657051    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657054    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657057    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657060    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657063    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657066    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657069    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:47:59.658837 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657071    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657832    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657840    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657843    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657847    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657850    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657853    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657857    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657860    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657864    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657867    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657869    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657873    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657875    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657878    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657881    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657883    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657886    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657888    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657891    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:47:59.659305 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657893    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657896    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657899    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657902    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657904    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657907    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657910    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657912    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657915    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657918    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657920    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657923    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657925    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657927    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657930    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657932    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657936    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657938    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657941    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657944    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:47:59.659947 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657947    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657949    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657952    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657954    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657957    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657959    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657961    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657964    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657966    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657969    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657971    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657974    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657976    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657979    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657982    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657985    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657987    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657990    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657992    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657995    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:47:59.660439 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.657998    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658002    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658006    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658008    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658011    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658013    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658016    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658019    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658021    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658024    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658026    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658029    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658031    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658034    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658037    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658039    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658042    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658044    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658047    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:47:59.660943 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658049    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658051    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658056    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658058    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658061    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658066    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658069    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658071    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658074    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658076    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658080    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658084    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658087    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658091    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658094    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658096    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658099    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658102    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658105    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658108    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:47:59.661411 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658110    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658588    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658595    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658598    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658601    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658604    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658613    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658616    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658619    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658621    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658624    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658627    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658630    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658633    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658657    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658661    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658664    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658667    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658669    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658672    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:47:59.661902 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658674    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658677    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658679    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658682    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658685    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658688    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658691    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658694    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658696    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658699    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658701    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658704    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658707    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658710    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658712    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658715    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658717    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658720    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658723    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:47:59.662400 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658731    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658734    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658736    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658739    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658741    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658744    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658746    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658750    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658753    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658756    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658758    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658761    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658763    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658766    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658768    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658771    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658773    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658776    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658781    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658785    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:47:59.662884 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658787    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658790    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658792    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658795    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658798    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658800    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658803    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658811    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658814    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658817    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658819    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658823    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658827    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658835    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658839    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658842    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658845    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658848    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658851    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658855    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:47:59.663369 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658858    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658861    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658864    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658866    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658869    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658871    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658874    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658877    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658879    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658882    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658885    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658887    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658890    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658892    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658896    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658899    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658903    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658905    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658908    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:47:59.663886 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658911    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.658914    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.658981    2563 flags.go:64] FLAG: --address="0.0.0.0"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659022    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659030    2563 flags.go:64] FLAG: --anonymous-auth="true"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659034    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659039    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659050    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659054    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659059    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659062    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659065    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659070    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659073    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659076    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659080    2563 flags.go:64] FLAG: --cgroup-root=""
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659082    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659086    2563 flags.go:64] FLAG: --client-ca-file=""
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659088    2563 flags.go:64] FLAG: --cloud-provider="external"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659091    2563 flags.go:64] FLAG: --cluster-dns="[]"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659098    2563 flags.go:64] FLAG: --cluster-domain=""
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659101    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659105    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659110    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 14 07:47:59.664349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659115    2563 flags.go:64] FLAG: --container-log-max-files="5"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659119    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659122    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659126    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659129    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659132    2563 flags.go:64] FLAG: --contention-profiling="false"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659135    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659138    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659142    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659145    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659149    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659152    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659154    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659157    2563 flags.go:64] FLAG: --enable-load-reader="false"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659160    2563 flags.go:64] FLAG: --enable-server="true"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659164    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659177    2563 flags.go:64] FLAG: --event-burst="100"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659180    2563 flags.go:64] FLAG: --event-qps="50"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659183    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659186    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659190    2563 flags.go:64] FLAG: --eviction-hard=""
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659194    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659197    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659201    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659204    2563 flags.go:64] FLAG: --eviction-soft=""
Sep 14 07:47:59.664936 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659207    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659210    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659212    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659215    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659218    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659221    2563 flags.go:64] FLAG: --fail-swap-on="true"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659224    2563 flags.go:64] FLAG: --feature-gates=""
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659228    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659231    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659234    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659237    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659240    2563 flags.go:64] FLAG: --healthz-port="10248"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659243    2563 flags.go:64] FLAG: --help="false"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659246    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-143-250.ec2.internal"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659250    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659253    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659256    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659260    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659263    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659266    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659269    2563 flags.go:64] FLAG: --image-service-endpoint=""
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659271    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659274    2563 flags.go:64] FLAG: --kube-api-burst="100"
Sep 14 07:47:59.665559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659277    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659285    2563 flags.go:64] FLAG: --kube-api-qps="50"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659289    2563 flags.go:64] FLAG: --kube-reserved=""
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659292    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659296    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659299    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659302    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659305    2563 flags.go:64] FLAG: --lock-file=""
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659308    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659311    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659315    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659324    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659327    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659330    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659333    2563 flags.go:64] FLAG: --logging-format="text"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659336    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659340    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659342    2563 flags.go:64] FLAG: --manifest-url=""
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659345    2563 flags.go:64] FLAG: --manifest-url-header=""
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659350    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659353    2563 flags.go:64] FLAG: --max-open-files="1000000"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659357    2563 flags.go:64] FLAG: --max-pods="110"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659360    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659365    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659368    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 14 07:47:59.666158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659371    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659374    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659377    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659380    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659387    2563 flags.go:64] FLAG: --node-status-max-images="50"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659390    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659393    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659396    2563 flags.go:64] FLAG: --pod-cidr=""
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659400    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659412    2563 flags.go:64] FLAG: --pod-manifest-path=""
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659416    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659419    2563 flags.go:64] FLAG: --pods-per-core="0"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659424    2563 flags.go:64] FLAG: --port="10250"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659427    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659430    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01f860d26c8a95635"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659434    2563 flags.go:64] FLAG: --qos-reserved=""
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659437    2563 flags.go:64] FLAG: --read-only-port="10255"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659440    2563 flags.go:64] FLAG: --register-node="true"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659442    2563 flags.go:64] FLAG: --register-with-taints=""
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659446    2563 flags.go:64] FLAG: --registry-burst="10"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659449    2563 flags.go:64] FLAG: --registry-qps="5"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659452    2563 flags.go:64] FLAG: --reserved-cpus=""
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659455    2563 flags.go:64] FLAG: --reserved-memory=""
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659459    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659462    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 14 07:47:59.666779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659465    2563 flags.go:64] FLAG: --rotate-certificates="false"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659468    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659471    2563 flags.go:64] FLAG: --runonce="false"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659474    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659477    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659481    2563 flags.go:64] FLAG: --seccomp-default="false"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659484    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659487    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659489    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659493    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659496    2563 flags.go:64] FLAG: --storage-driver-password="root"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659498    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659501    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659504    2563 flags.go:64] FLAG: --storage-driver-user="root"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659507    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659510    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659513    2563 flags.go:64] FLAG: --system-cgroups=""
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659521    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659526    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659530    2563 flags.go:64] FLAG: --tls-cert-file=""
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659533    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659552    2563 flags.go:64] FLAG: --tls-min-version=""
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659556    2563 flags.go:64] FLAG: --tls-private-key-file=""
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659559    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659563    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 14 07:47:59.667382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659566    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659569    2563 flags.go:64] FLAG: --v="2"
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659574    2563 flags.go:64] FLAG: --version="false"
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659578    2563 flags.go:64] FLAG: --vmodule=""
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659581    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.659585    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659755    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659761    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659765    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659768    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659771    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659774    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659779    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659781    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659784    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659787    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659790    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659793    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659795    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659798    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659800    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:47:59.668023 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659803    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659805    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659808    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659811    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659819    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659823    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659826    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659829    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659831    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659834    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659836    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659839    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659841    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659844    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659847    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659849    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659852    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659854    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659857    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659860    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:47:59.668814 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659862    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659865    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659867    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659871    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659873    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659876    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659878    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659881    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659884    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659886    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659889    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659892    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659896    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659899    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659902    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659904    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659908    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659920    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659923    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:47:59.669448 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659926    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659929    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659931    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659934    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659936    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659939    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659942    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659944    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659947    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659950    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659953    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659956    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659959    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659961    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659964    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659967    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659971    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659974    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659976    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:47:59.670019 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659979    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659981    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659984    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659987    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659989    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659992    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659994    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659997    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.659999    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660002    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660005    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660010    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660020    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660023    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660026    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660028    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660031    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660033    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660036    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660038    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:47:59.670487 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660041    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:47:59.671035 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660044    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:47:59.671035 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660046    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:47:59.671035 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660049    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:47:59.671035 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660051    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:47:59.671035 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.660054    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:47:59.671035 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.660775    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:47:59.671035 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.670166    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 14 07:47:59.671035 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.670182    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.671054    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.671063    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671121    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671125    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671129    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671132    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671135    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671138    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671141    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671144    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671147    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671149    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671152    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671155    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671160    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671165    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671169    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:47:59.671242 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671172    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671175    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671177    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671180    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671183    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671186    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671189    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671191    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671194    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671196    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671199    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671203    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671206    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671210    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671213    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671215    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671220    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671223    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671226    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:47:59.671719 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671228    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671232    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671235    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671238    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671240    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671243    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671246    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671248    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671251    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671254    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671256    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671259    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671262    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671264    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671267    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671270    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671272    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671275    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671278    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671280    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:47:59.672192 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671283    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671285    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671287    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671290    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671292    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671295    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671297    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671300    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671303    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671306    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671309    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671311    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671313    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671316    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671319    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671322    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671324    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671327    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671329    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671332    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:47:59.672763 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671334    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671337    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671339    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671342    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671344    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671348    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671350    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671353    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671355    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671358    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671360    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671363    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671365    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671367    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671370    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671373    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671376    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671378    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671381    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671383    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:47:59.673288 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671386    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671389    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671392    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671395    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671397    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.671402    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671512    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671517    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671520    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671523    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671526    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671528    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671532    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671537    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:47:59.673786 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671540    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671543    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671546    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671548    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671552    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671554    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671557    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671559    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671562    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671565    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671567    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671570    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671573    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671576    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671578    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671581    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671584    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671587    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671589    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671592    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:47:59.674136 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671595    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671598    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671601    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671603    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671606    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671609    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671611    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671614    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671617    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671619    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671622    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671625    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671627    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671630    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671653    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671657    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671659    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671663    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671667    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671671    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:47:59.674621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671674    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671676    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671679    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671682    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671685    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671688    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671690    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671693    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671696    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671698    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671700    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671703    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671706    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671709    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671712    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671714    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671717    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671719    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671722    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671725    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:47:59.675112 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671728    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671731    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671733    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671736    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671739    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671741    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671744    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671746    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671748    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671751    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671755    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671757    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671760    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671762    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671765    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671768    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671771    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671773    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671776    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671779    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:47:59.675588 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671781    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:47:59.676102 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671784    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:47:59.676102 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671786    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:47:59.676102 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671789    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:47:59.676102 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671792    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:47:59.676102 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671794    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:47:59.676102 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671797    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:47:59.676102 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671800    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:47:59.676102 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671802    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:47:59.676102 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671805    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:47:59.676102 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:47:59.671809    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:47:59.676102 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.671814    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:47:59.676102 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.671912    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 14 07:47:59.676697 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.676683    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 14 07:47:59.677701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.677690    2563 server.go:1019] "Starting client certificate rotation"
Sep 14 07:47:59.677811 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.677793    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 07:47:59.677848 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.677831    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 07:47:59.680735 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.680684    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 07:47:59.682402 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.682384    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 07:47:59.692223 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.692208    2563 log.go:25] "Validated CRI v1 runtime API"
Sep 14 07:47:59.697789 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.697774    2563 log.go:25] "Validated CRI v1 image API"
Sep 14 07:47:59.699480 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.699467    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 14 07:47:59.702069 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.702049    2563 fs.go:135] Filesystem UUIDs: map[6ba9c408-bdc7-419d-9321-43462495ff22:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ed71c9b3-6493-41e3-bf40-9bbe973ac69e:/dev/nvme0n1p4]
Sep 14 07:47:59.702155 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.702068    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 14 07:47:59.709381 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.709262    2563 manager.go:216] Machine: {Timestamp:2026-09-14 07:47:59.70738058 +0000 UTC m=+0.336268232 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093581 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20c145ace939fa6ffc21f1b422f5b5 SystemUUID:ec20c145-ace9-39fa-6ffc-21f1b422f5b5 BootID:10564746-4964-4143-a3ca-f55cd8439570 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d7:52:4b:0b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d7:52:4b:0b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:fc:9b:28:42:91 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 14 07:47:59.709381 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.709372    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 14 07:47:59.709513 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.709447    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 14 07:47:59.710534 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.710520    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 07:47:59.712865 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.712840    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 14 07:47:59.712996 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.712868    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-250.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 14 07:47:59.714457 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.714446    2563 topology_manager.go:138] "Creating topology manager with none policy"
Sep 14 07:47:59.714491 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.714460    2563 container_manager_linux.go:309] "Creating device plugin manager"
Sep 14 07:47:59.714491 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.714468    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 07:47:59.715268 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.715258    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 07:47:59.717198 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.717187    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 14 07:47:59.721036 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.721024    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 07:47:59.721143 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.721134    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 14 07:47:59.721175 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.721163    2563 kubelet.go:486] "Attempting to sync node with API server"
Sep 14 07:47:59.721175 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.721173    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 14 07:47:59.721237 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.721184    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 14 07:47:59.721237 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.721193    2563 kubelet.go:398] "Adding apiserver pod source"
Sep 14 07:47:59.721237 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.721201    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 14 07:47:59.721570 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.721557    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 07:47:59.721570 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.721577    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 07:47:59.723938 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.723922    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 14 07:47:59.727039 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.727026    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 14 07:47:59.727100 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.727056    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 14 07:47:59.729022 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.729009    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 14 07:47:59.729022 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.729024    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 14 07:47:59.729118 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.729031    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 14 07:47:59.729118 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.729036    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 14 07:47:59.729118 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.729045    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 14 07:47:59.729118 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.729050    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 14 07:47:59.729118 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.729063    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 14 07:47:59.729118 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.729069    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 14 07:47:59.729118 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.729076    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 14 07:47:59.729118 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.729081    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 14 07:47:59.729118 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.729090    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 14 07:47:59.729118 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.729095    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 14 07:47:59.729954 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:47:59.729934    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 14 07:47:59.730037 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.730027    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 14 07:47:59.730037 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.730036    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 14 07:47:59.730126 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:47:59.730109    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-250.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 14 07:47:59.732917 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.732894    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-250.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 14 07:47:59.733878 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.733865    2563 server.go:1268] "Started kubelet"
Sep 14 07:47:59.733931 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.733912    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 14 07:47:59.734124 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.734074    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 14 07:47:59.734182 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.734157    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 14 07:47:59.734782 ip-10-0-143-250 systemd[1]: Started Kubernetes Kubelet.
Sep 14 07:47:59.734992 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.734974    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n4r9r"
Sep 14 07:47:59.737306 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.737289    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 14 07:47:59.737531 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.737504    2563 server.go:310] "Adding debug handlers to kubelet server"
Sep 14 07:47:59.738472 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:47:59.738440    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 14 07:47:59.742815 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.742799    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n4r9r"
Sep 14 07:47:59.742815 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:47:59.741624    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-250.ec2.internal.18d5206d0f8988dd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-250.ec2.internal,UID:ip-10-0-143-250.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-250.ec2.internal,},FirstTimestamp:2026-09-14 07:47:59.733835997 +0000 UTC m=+0.362723615,LastTimestamp:2026-09-14 07:47:59.733835997 +0000 UTC m=+0.362723615,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-250.ec2.internal,}"
Sep 14 07:47:59.743946 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.743931    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 14 07:47:59.744627 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.744611    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 14 07:47:59.744782 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:47:59.744765    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-250.ec2.internal\" not found"
Sep 14 07:47:59.745361 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.745335    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 14 07:47:59.745361 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.745335    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 14 07:47:59.745486 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.745368    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 14 07:47:59.745486 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.745469    2563 factory.go:55] Registering systemd factory
Sep 14 07:47:59.745486 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.745480    2563 factory.go:223] Registration of the systemd container factory successfully
Sep 14 07:47:59.745619 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.745479    2563 reconstruct.go:97] "Volume reconstruction finished"
Sep 14 07:47:59.745619 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.745507    2563 reconciler.go:29] "Reconciler: start to sync state"
Sep 14 07:47:59.745725 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.745675    2563 factory.go:153] Registering CRI-O factory
Sep 14 07:47:59.745725 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.745685    2563 factory.go:223] Registration of the crio container factory successfully
Sep 14 07:47:59.745725 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.745723    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 14 07:47:59.745850 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.745736    2563 factory.go:103] Registering Raw factory
Sep 14 07:47:59.745850 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.745748    2563 manager.go:1196] Started watching for new ooms in manager
Sep 14 07:47:59.746024 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.746009    2563 manager.go:349] Starting recovery of all containers
Sep 14 07:47:59.748408 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:47:59.748380    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 14 07:47:59.748522 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:47:59.748444    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-250.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 14 07:47:59.758337 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.758320    2563 manager.go:354] Recovery completed
Sep 14 07:47:59.763025 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.763012    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:59.765348 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.765329    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:59.765403 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.765364    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:59.765403 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.765375    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:47:59.766012 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.765995    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 14 07:47:59.766012 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.766008    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:59.766123 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.766010    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 14 07:47:59.766123 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.766044    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 07:47:59.766769 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.766754    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:59.766830 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.766781    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:59.766830 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.766795    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:47:59.767158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.767146    2563 policy_none.go:49] "None policy: Start"
Sep 14 07:47:59.767190 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.767162    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 14 07:47:59.767190 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.767172    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 14 07:47:59.768058 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.768048    2563 policy_none.go:47] "Start"
Sep 14 07:47:59.813251 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.813237    2563 manager.go:337] "Starting Device Plugin manager"
Sep 14 07:47:59.815329 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:47:59.813329    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 14 07:47:59.815329 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.813345    2563 server.go:87] "Starting device plugin registration server"
Sep 14 07:47:59.815329 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.813584    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 14 07:47:59.815329 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.813613    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 14 07:47:59.815329 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.813784    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 14 07:47:59.815329 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.813864    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 14 07:47:59.815329 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.813873    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 14 07:47:59.815329 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:47:59.814430    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 14 07:47:59.815329 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:47:59.814567    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-250.ec2.internal\" not found"
Sep 14 07:47:59.877841 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.877791    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 14 07:47:59.879050 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.879032    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 14 07:47:59.879153 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.879089    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 14 07:47:59.879153 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.879114    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 14 07:47:59.879261 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:47:59.879200    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 14 07:47:59.882079 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.882058    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:47:59.914499 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.914442    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:59.915262 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.915247    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:59.915321 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.915275    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:59.915321 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.915286    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:47:59.915321 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.915308    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-250.ec2.internal"
Sep 14 07:47:59.923056 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.923039    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-250.ec2.internal"
Sep 14 07:47:59.923116 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:47:59.923064    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-250.ec2.internal\": node \"ip-10-0-143-250.ec2.internal\" not found"
Sep 14 07:47:59.938972 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:47:59.938953    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-250.ec2.internal\" not found"
Sep 14 07:47:59.979843 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.979819    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-250.ec2.internal"]
Sep 14 07:47:59.979922 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.979885    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:59.981700 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.981684    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:59.981759 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.981714    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:59.981759 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.981724    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:47:59.982838 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.982826    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:59.982994 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.982980    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal"
Sep 14 07:47:59.983041 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.983014    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:59.984010 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.983996    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:59.984070 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.984019    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:59.984070 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.984028    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:47:59.984070 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.984041    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:59.984070 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.984059    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:59.984207 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.984073    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:47:59.985334 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.985317    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-250.ec2.internal"
Sep 14 07:47:59.985412 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.985355    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:59.986149 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.986123    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:59.986236 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.986152    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:59.986236 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:47:59.986169    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:48:00.009191 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:00.009171    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-250.ec2.internal\" not found" node="ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.013541 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:00.013520    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-250.ec2.internal\" not found" node="ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.039501 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:00.039482    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-250.ec2.internal\" not found"
Sep 14 07:48:00.046756 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.046736    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ecb1680172f83e440b8fb7199dfde94a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal\" (UID: \"ecb1680172f83e440b8fb7199dfde94a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.046819 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.046761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ecb1680172f83e440b8fb7199dfde94a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal\" (UID: \"ecb1680172f83e440b8fb7199dfde94a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.046819 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.046778    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/520462c720d404adb820140db50e2930-config\") pod \"kube-apiserver-proxy-ip-10-0-143-250.ec2.internal\" (UID: \"520462c720d404adb820140db50e2930\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.140432 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:00.140403    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-250.ec2.internal\" not found"
Sep 14 07:48:00.147745 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.147722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ecb1680172f83e440b8fb7199dfde94a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal\" (UID: \"ecb1680172f83e440b8fb7199dfde94a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.147832 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.147754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ecb1680172f83e440b8fb7199dfde94a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal\" (UID: \"ecb1680172f83e440b8fb7199dfde94a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.147832 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.147778    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/520462c720d404adb820140db50e2930-config\") pod \"kube-apiserver-proxy-ip-10-0-143-250.ec2.internal\" (UID: \"520462c720d404adb820140db50e2930\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.147832 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.147820    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ecb1680172f83e440b8fb7199dfde94a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal\" (UID: \"ecb1680172f83e440b8fb7199dfde94a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.147951 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.147835    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/520462c720d404adb820140db50e2930-config\") pod \"kube-apiserver-proxy-ip-10-0-143-250.ec2.internal\" (UID: \"520462c720d404adb820140db50e2930\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.147951 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.147820    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ecb1680172f83e440b8fb7199dfde94a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal\" (UID: \"ecb1680172f83e440b8fb7199dfde94a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.241220 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:00.241140    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-250.ec2.internal\" not found"
Sep 14 07:48:00.311736 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.311694    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.316586 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.316561    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.342137 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:00.342115    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-250.ec2.internal\" not found"
Sep 14 07:48:00.444933 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:00.444878    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-250.ec2.internal\" not found"
Sep 14 07:48:00.545198 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:00.545086    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-250.ec2.internal\" not found"
Sep 14 07:48:00.577320 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.577286    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:48:00.649380 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:00.649347    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-250.ec2.internal\" not found"
Sep 14 07:48:00.677782 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.677757    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 14 07:48:00.678466 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.677896    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 07:48:00.678466 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.677949    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 07:48:00.748326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.748277    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-13 07:43:00 +0000 UTC" deadline="2028-03-12 06:48:20.50482451 +0000 UTC"
Sep 14 07:48:00.748326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.748321    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13079h0m19.756507009s"
Sep 14 07:48:00.749768 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:00.749749    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-250.ec2.internal\" not found"
Sep 14 07:48:00.850422 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:00.850346    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-250.ec2.internal\" not found"
Sep 14 07:48:00.862221 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:00.862172    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecb1680172f83e440b8fb7199dfde94a.slice/crio-2e403ebe0786bf579013f051536f13b5eaf2be39005f5bd89daea5b513bb1582 WatchSource:0}: Error finding container 2e403ebe0786bf579013f051536f13b5eaf2be39005f5bd89daea5b513bb1582: Status 404 returned error can't find the container with id 2e403ebe0786bf579013f051536f13b5eaf2be39005f5bd89daea5b513bb1582
Sep 14 07:48:00.864253 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:00.864217    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod520462c720d404adb820140db50e2930.slice/crio-9db6e1d0b6f08e237bf4ee7043d95cbae90ffaec36d7fff07966fdc5de6839b3 WatchSource:0}: Error finding container 9db6e1d0b6f08e237bf4ee7043d95cbae90ffaec36d7fff07966fdc5de6839b3: Status 404 returned error can't find the container with id 9db6e1d0b6f08e237bf4ee7043d95cbae90ffaec36d7fff07966fdc5de6839b3
Sep 14 07:48:00.866844 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.866749    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:48:00.867404 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.867386    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:48:00.882970 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.882912    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-250.ec2.internal" event={"ID":"520462c720d404adb820140db50e2930","Type":"ContainerStarted","Data":"9db6e1d0b6f08e237bf4ee7043d95cbae90ffaec36d7fff07966fdc5de6839b3"}
Sep 14 07:48:00.884201 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.884181    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal" event={"ID":"ecb1680172f83e440b8fb7199dfde94a","Type":"ContainerStarted","Data":"2e403ebe0786bf579013f051536f13b5eaf2be39005f5bd89daea5b513bb1582"}
Sep 14 07:48:00.939762 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.939736    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:48:00.944948 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.944921    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.956556 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.956535    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 07:48:00.957733 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.957711    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-250.ec2.internal"
Sep 14 07:48:00.967134 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:00.967116    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 07:48:01.723385 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.723349    2563 apiserver.go:52] "Watching apiserver"
Sep 14 07:48:01.732753 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.731939    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 14 07:48:01.732753 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.732667    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-zwg2q","openshift-dns/node-resolver-ts5z9","openshift-multus/network-metrics-daemon-46qgp","openshift-network-diagnostics/network-check-target-vpz6b","openshift-network-operator/iptables-alerter-wszct","openshift-ovn-kubernetes/ovnkube-node-w4r2f","kube-system/konnectivity-agent-ffvbt","kube-system/kube-apiserver-proxy-ip-10-0-143-250.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c","openshift-image-registry/node-ca-r4h2b","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal","openshift-multus/multus-7z55c","openshift-multus/multus-additional-cni-plugins-v22pk","kube-system/global-pull-secret-syncer-zwmpr"]
Sep 14 07:48:01.735324 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.734792    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ffvbt"
Sep 14 07:48:01.736721 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.735987    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.737170 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.737136    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:01.737276 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:01.737214    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:01.737562 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.737530    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 14 07:48:01.737562 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.737533    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n56s4\""
Sep 14 07:48:01.737731 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.737572    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 14 07:48:01.738539 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.738515    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:01.738625 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:01.738589    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:01.739036 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.739017    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 14 07:48:01.739116 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.739036    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fmbvn\""
Sep 14 07:48:01.739321 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.739304    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 14 07:48:01.739426 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.739407    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 14 07:48:01.739543 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.739528    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 14 07:48:01.739903 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.739744    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 14 07:48:01.740701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.740636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wszct"
Sep 14 07:48:01.741825 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.741805    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.743661 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.743323    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 14 07:48:01.743661 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.743362    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:48:01.743661 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.743379    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-562nk\""
Sep 14 07:48:01.743661 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.743397    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 14 07:48:01.745093 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.744844    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 14 07:48:01.745093 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.744850    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 14 07:48:01.745093 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.745039    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 14 07:48:01.745261 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.745244    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 14 07:48:01.745337 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.745320    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 14 07:48:01.745472 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.745457    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 14 07:48:01.745520 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.745489    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dzrlc\""
Sep 14 07:48:01.745637 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.745620    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 14 07:48:01.746766 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.746325    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.747961 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.747891    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.749530 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.749510    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mb5cd\""
Sep 14 07:48:01.749919 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.749902    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 14 07:48:01.750619 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.750594    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4nmtm\""
Sep 14 07:48:01.751737 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.751696    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 14 07:48:01.751829 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.751771    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:48:01.751894 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.751837    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 14 07:48:01.752559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.752532    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ts5z9"
Sep 14 07:48:01.753451 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.753430    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 14 07:48:01.754925 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.754888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hrp8\" (UniqueName: \"kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8\") pod \"network-check-target-vpz6b\" (UID: \"e47647c7-20f1-440e-83da-65828c7f4605\") " pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:01.755023 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.754936    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.755023 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.754973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c0fb32a4-26e3-4097-9577-6c7057da8697-ovnkube-script-lib\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.755023 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8d1ffd7-116f-4af9-bf09-5e3df09c4155-host-slash\") pod \"iptables-alerter-wszct\" (UID: \"a8d1ffd7-116f-4af9-bf09-5e3df09c4155\") " pod="openshift-network-operator/iptables-alerter-wszct"
Sep 14 07:48:01.755189 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-etc-selinux\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.755189 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-sys-fs\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.755189 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9f00ee5-9070-420e-9fe1-3e5e0482610e-cni-binary-copy\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.755189 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755178    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9f00ee5-9070-420e-9fe1-3e5e0482610e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.755388 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755213    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:01.755388 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755246    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-run-openvswitch\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.755388 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755271    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 14 07:48:01.755388 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755311    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lr4kj\""
Sep 14 07:48:01.755585 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755273    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-run-ovn\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.755585 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-socket-dir\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.755585 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-etc-openvswitch\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.755825 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c0fb32a4-26e3-4097-9577-6c7057da8697-env-overrides\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.755825 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755596    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 14 07:48:01.755825 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.755634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c0fb32a4-26e3-4097-9577-6c7057da8697-ovn-node-metrics-cert\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.756748 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.756724    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r4h2b"
Sep 14 07:48:01.759631 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.756049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a8d1ffd7-116f-4af9-bf09-5e3df09c4155-iptables-alerter-script\") pod \"iptables-alerter-wszct\" (UID: \"a8d1ffd7-116f-4af9-bf09-5e3df09c4155\") " pod="openshift-network-operator/iptables-alerter-wszct"
Sep 14 07:48:01.759744 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.759662    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-device-dir\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.759744 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.759709    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61f94071-3e09-4079-abb1-bbfa74ae8453-konnectivity-ca\") pod \"konnectivity-agent-ffvbt\" (UID: \"61f94071-3e09-4079-abb1-bbfa74ae8453\") " pod="kube-system/konnectivity-agent-ffvbt"
Sep 14 07:48:01.759744 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.759738    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-log-socket\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.759915 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.759775    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-cni-bin\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.759915 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.759803    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c0fb32a4-26e3-4097-9577-6c7057da8697-ovnkube-config\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.759915 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.759829    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-registration-dir\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.759915 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.759856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9f00ee5-9070-420e-9fe1-3e5e0482610e-system-cni-dir\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.759915 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.759906    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgvwv\" (UniqueName: \"kubernetes.io/projected/d9f00ee5-9070-420e-9fe1-3e5e0482610e-kube-api-access-qgvwv\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.760169 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.759933    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-cni-netd\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.760169 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.759957    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9th9h\" (UniqueName: \"kubernetes.io/projected/a8d1ffd7-116f-4af9-bf09-5e3df09c4155-kube-api-access-9th9h\") pod \"iptables-alerter-wszct\" (UID: \"a8d1ffd7-116f-4af9-bf09-5e3df09c4155\") " pod="openshift-network-operator/iptables-alerter-wszct"
Sep 14 07:48:01.760169 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.759982    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c7jz\" (UniqueName: \"kubernetes.io/projected/17454c80-e6af-4f63-bc8c-84a6f1cb536b-kube-api-access-6c7jz\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.760169 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760078    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-run-netns\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.760169 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-var-lib-openvswitch\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.760169 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-node-log\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.760169 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760166    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-run-ovn-kubernetes\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.760592 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gng7\" (UniqueName: \"kubernetes.io/projected/c0fb32a4-26e3-4097-9577-6c7057da8697-kube-api-access-2gng7\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.760592 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.760592 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61f94071-3e09-4079-abb1-bbfa74ae8453-agent-certs\") pod \"konnectivity-agent-ffvbt\" (UID: \"61f94071-3e09-4079-abb1-bbfa74ae8453\") " pod="kube-system/konnectivity-agent-ffvbt"
Sep 14 07:48:01.760592 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760290    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9f00ee5-9070-420e-9fe1-3e5e0482610e-cnibin\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.760592 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-slash\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.760592 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-run-systemd\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.760592 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760373    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9f00ee5-9070-420e-9fe1-3e5e0482610e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.760592 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-kubelet\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.760592 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760435    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-systemd-units\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.761135 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760625    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 14 07:48:01.761135 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760633    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9f00ee5-9070-420e-9fe1-3e5e0482610e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.761135 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.760666    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hsrk\" (UniqueName: \"kubernetes.io/projected/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-kube-api-access-8hsrk\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:01.761975 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.761946    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 14 07:48:01.762540 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.762235    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jjn58\""
Sep 14 07:48:01.762540 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.762291    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:01.762540 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:01.762390    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:01.762540 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.762439    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 14 07:48:01.762816 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.762698    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 07:48:01.763407 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.763378    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.765421 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.765401    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 14 07:48:01.765746 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.765729    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w922p\""
Sep 14 07:48:01.783321 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.783228    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s624p"
Sep 14 07:48:01.793728 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.793686    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s624p"
Sep 14 07:48:01.846913 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.846874    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 14 07:48:01.862000 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.861950    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9f00ee5-9070-420e-9fe1-3e5e0482610e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.862191 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5zfx\" (UniqueName: \"kubernetes.io/projected/261155e3-6b52-449b-94ca-649f7210ccd9-kube-api-access-q5zfx\") pod \"node-resolver-ts5z9\" (UID: \"261155e3-6b52-449b-94ca-649f7210ccd9\") " pod="openshift-dns/node-resolver-ts5z9"
Sep 14 07:48:01.862191 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9f00ee5-9070-420e-9fe1-3e5e0482610e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.862191 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862082    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hsrk\" (UniqueName: \"kubernetes.io/projected/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-kube-api-access-8hsrk\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:01.862191 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.862191 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c0fb32a4-26e3-4097-9577-6c7057da8697-ovnkube-script-lib\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.862191 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-var-lib-cni-multus\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.862510 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862207    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7v8h\" (UniqueName: \"kubernetes.io/projected/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-kube-api-access-l7v8h\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.862510 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/261155e3-6b52-449b-94ca-649f7210ccd9-hosts-file\") pod \"node-resolver-ts5z9\" (UID: \"261155e3-6b52-449b-94ca-649f7210ccd9\") " pod="openshift-dns/node-resolver-ts5z9"
Sep 14 07:48:01.862510 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/261155e3-6b52-449b-94ca-649f7210ccd9-tmp-dir\") pod \"node-resolver-ts5z9\" (UID: \"261155e3-6b52-449b-94ca-649f7210ccd9\") " pod="openshift-dns/node-resolver-ts5z9"
Sep 14 07:48:01.862510 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-sys-fs\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.862510 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9f00ee5-9070-420e-9fe1-3e5e0482610e-cni-binary-copy\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.862510 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862338    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:01.862510 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862378    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c0fb32a4-26e3-4097-9577-6c7057da8697-ovn-node-metrics-cert\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.862510 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862403    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-sysconfig\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.862510 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bee2d6cd-aca8-4008-be09-34cd9916d966-kubelet-config\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:01.862510 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-socket-dir\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.862510 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-etc-openvswitch\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862524    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-var-lib-cni-bin\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862549    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-var-lib-kubelet\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862574    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-device-dir\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61f94071-3e09-4079-abb1-bbfa74ae8453-konnectivity-ca\") pod \"konnectivity-agent-ffvbt\" (UID: \"61f94071-3e09-4079-abb1-bbfa74ae8453\") " pod="kube-system/konnectivity-agent-ffvbt"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-log-socket\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862675    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-tuned\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862707    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-registration-dir\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9f00ee5-9070-420e-9fe1-3e5e0482610e-system-cni-dir\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862778    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvwv\" (UniqueName: \"kubernetes.io/projected/d9f00ee5-9070-420e-9fe1-3e5e0482610e-kube-api-access-qgvwv\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9th9h\" (UniqueName: \"kubernetes.io/projected/a8d1ffd7-116f-4af9-bf09-5e3df09c4155-kube-api-access-9th9h\") pod \"iptables-alerter-wszct\" (UID: \"a8d1ffd7-116f-4af9-bf09-5e3df09c4155\") " pod="openshift-network-operator/iptables-alerter-wszct"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862830    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-run-netns\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862855    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-node-log\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-run-ovn-kubernetes\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862922    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-system-cni-dir\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-var-lib-kubelet\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.862971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-run-multus-certs\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.863127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-slash\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-multus-cni-dir\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-hostroot\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863112    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-host\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-kubelet\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-systemd-units\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863189    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-cnibin\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863212    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-sys\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjvs5\" (UniqueName: \"kubernetes.io/projected/1a1a3710-8da0-4352-9225-64e4ebac48fd-kube-api-access-zjvs5\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrp8\" (UniqueName: \"kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8\") pod \"network-check-target-vpz6b\" (UID: \"e47647c7-20f1-440e-83da-65828c7f4605\") " pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8d1ffd7-116f-4af9-bf09-5e3df09c4155-host-slash\") pod \"iptables-alerter-wszct\" (UID: \"a8d1ffd7-116f-4af9-bf09-5e3df09c4155\") " pod="openshift-network-operator/iptables-alerter-wszct"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-multus-conf-dir\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863398    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-multus-daemon-config\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863423    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-sysctl-conf\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863447    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a1a3710-8da0-4352-9225-64e4ebac48fd-tmp\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863469    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bee2d6cd-aca8-4008-be09-34cd9916d966-dbus\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863496    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:01.863892 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-etc-selinux\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9f00ee5-9070-420e-9fe1-3e5e0482610e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-run-openvswitch\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863627    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-run-ovn\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863637    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c0fb32a4-26e3-4097-9577-6c7057da8697-ovnkube-script-lib\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863658    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a8d1ffd7-116f-4af9-bf09-5e3df09c4155-iptables-alerter-script\") pod \"iptables-alerter-wszct\" (UID: \"a8d1ffd7-116f-4af9-bf09-5e3df09c4155\") " pod="openshift-network-operator/iptables-alerter-wszct"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-cni-binary-copy\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-modprobe-d\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c0fb32a4-26e3-4097-9577-6c7057da8697-env-overrides\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-lib-modules\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863821    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9f00ee5-9070-420e-9fe1-3e5e0482610e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-cni-bin\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c0fb32a4-26e3-4097-9577-6c7057da8697-ovnkube-config\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-multus-socket-dir-parent\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863958    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-run-k8s-cni-cncf-io\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-systemd\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863997    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq4hg\" (UniqueName: \"kubernetes.io/projected/416afa6c-5c55-487f-8de6-200bc0be3e06-kube-api-access-nq4hg\") pod \"node-ca-r4h2b\" (UID: \"416afa6c-5c55-487f-8de6-200bc0be3e06\") " pod="openshift-image-registry/node-ca-r4h2b"
Sep 14 07:48:01.864553 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.863996    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-cni-netd\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-etc-kubernetes\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/416afa6c-5c55-487f-8de6-200bc0be3e06-host\") pod \"node-ca-r4h2b\" (UID: \"416afa6c-5c55-487f-8de6-200bc0be3e06\") " pod="openshift-image-registry/node-ca-r4h2b"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:01.864066    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:01.864129    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs podName:c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:02.364110206 +0000 UTC m=+2.992997817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs") pod "network-metrics-daemon-46qgp" (UID: "c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864228    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-registration-dir\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864244    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9f00ee5-9070-420e-9fe1-3e5e0482610e-system-cni-dir\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864313    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-kubelet\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864348    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-systemd-units\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864517    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8d1ffd7-116f-4af9-bf09-5e3df09c4155-host-slash\") pod \"iptables-alerter-wszct\" (UID: \"a8d1ffd7-116f-4af9-bf09-5e3df09c4155\") " pod="openshift-network-operator/iptables-alerter-wszct"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864608    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-etc-selinux\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-run-netns\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-run-openvswitch\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864762    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-run-ovn\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-sys-fs\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.865323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-run-ovn-kubernetes\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864926    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-node-log\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864963    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-socket-dir\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865021    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-device-dir\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865063    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-cni-netd\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865318    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c0fb32a4-26e3-4097-9577-6c7057da8697-ovnkube-config\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c0fb32a4-26e3-4097-9577-6c7057da8697-env-overrides\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.864068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c7jz\" (UniqueName: \"kubernetes.io/projected/17454c80-e6af-4f63-bc8c-84a6f1cb536b-kube-api-access-6c7jz\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9f00ee5-9070-420e-9fe1-3e5e0482610e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865562    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/61f94071-3e09-4079-abb1-bbfa74ae8453-konnectivity-ca\") pod \"konnectivity-agent-ffvbt\" (UID: \"61f94071-3e09-4079-abb1-bbfa74ae8453\") " pod="kube-system/konnectivity-agent-ffvbt"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-cni-bin\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865613    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-etc-openvswitch\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865613    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-log-socket\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865620    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-host-slash\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-var-lib-openvswitch\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gng7\" (UniqueName: \"kubernetes.io/projected/c0fb32a4-26e3-4097-9577-6c7057da8697-kube-api-access-2gng7\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865755    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a8d1ffd7-116f-4af9-bf09-5e3df09c4155-iptables-alerter-script\") pod \"iptables-alerter-wszct\" (UID: \"a8d1ffd7-116f-4af9-bf09-5e3df09c4155\") " pod="openshift-network-operator/iptables-alerter-wszct"
Sep 14 07:48:01.866073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865772    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-var-lib-openvswitch\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-kubernetes\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-sysctl-d\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865837    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9f00ee5-9070-420e-9fe1-3e5e0482610e-cni-binary-copy\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/416afa6c-5c55-487f-8de6-200bc0be3e06-serviceca\") pod \"node-ca-r4h2b\" (UID: \"416afa6c-5c55-487f-8de6-200bc0be3e06\") " pod="openshift-image-registry/node-ca-r4h2b"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.865965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.866006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61f94071-3e09-4079-abb1-bbfa74ae8453-agent-certs\") pod \"konnectivity-agent-ffvbt\" (UID: \"61f94071-3e09-4079-abb1-bbfa74ae8453\") " pod="kube-system/konnectivity-agent-ffvbt"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.866039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9f00ee5-9070-420e-9fe1-3e5e0482610e-cnibin\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.866068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-run-systemd\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.866108    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/17454c80-e6af-4f63-bc8c-84a6f1cb536b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.866112    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-run-netns\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.866131    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c0fb32a4-26e3-4097-9577-6c7057da8697-run-systemd\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.866149    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9f00ee5-9070-420e-9fe1-3e5e0482610e-cnibin\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.866177    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-run\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.866893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.866177    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9f00ee5-9070-420e-9fe1-3e5e0482610e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.869201 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.869160    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c0fb32a4-26e3-4097-9577-6c7057da8697-ovn-node-metrics-cert\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.870055 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.870032    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/61f94071-3e09-4079-abb1-bbfa74ae8453-agent-certs\") pod \"konnectivity-agent-ffvbt\" (UID: \"61f94071-3e09-4079-abb1-bbfa74ae8453\") " pod="kube-system/konnectivity-agent-ffvbt"
Sep 14 07:48:01.876884 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:01.875870    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:01.876884 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:01.875898    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:01.876884 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:01.875911    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8hrp8 for pod openshift-network-diagnostics/network-check-target-vpz6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:01.876884 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:01.875985    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8 podName:e47647c7-20f1-440e-83da-65828c7f4605 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:02.37596332 +0000 UTC m=+3.004850927 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8hrp8" (UniqueName: "kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8") pod "network-check-target-vpz6b" (UID: "e47647c7-20f1-440e-83da-65828c7f4605") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:01.876884 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.876666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hsrk\" (UniqueName: \"kubernetes.io/projected/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-kube-api-access-8hsrk\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:01.878739 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.878317    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gng7\" (UniqueName: \"kubernetes.io/projected/c0fb32a4-26e3-4097-9577-6c7057da8697-kube-api-access-2gng7\") pod \"ovnkube-node-w4r2f\" (UID: \"c0fb32a4-26e3-4097-9577-6c7057da8697\") " pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:01.878739 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.878622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9th9h\" (UniqueName: \"kubernetes.io/projected/a8d1ffd7-116f-4af9-bf09-5e3df09c4155-kube-api-access-9th9h\") pod \"iptables-alerter-wszct\" (UID: \"a8d1ffd7-116f-4af9-bf09-5e3df09c4155\") " pod="openshift-network-operator/iptables-alerter-wszct"
Sep 14 07:48:01.879289 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.879267    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c7jz\" (UniqueName: \"kubernetes.io/projected/17454c80-e6af-4f63-bc8c-84a6f1cb536b-kube-api-access-6c7jz\") pod \"aws-ebs-csi-driver-node-wjb9c\" (UID: \"17454c80-e6af-4f63-bc8c-84a6f1cb536b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:01.880997 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.880971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgvwv\" (UniqueName: \"kubernetes.io/projected/d9f00ee5-9070-420e-9fe1-3e5e0482610e-kube-api-access-qgvwv\") pod \"multus-additional-cni-plugins-v22pk\" (UID: \"d9f00ee5-9070-420e-9fe1-3e5e0482610e\") " pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:01.967063 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-lib-modules\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.967063 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967069    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-multus-socket-dir-parent\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.967326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967100    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-run-k8s-cni-cncf-io\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.967326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-systemd\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.967326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq4hg\" (UniqueName: \"kubernetes.io/projected/416afa6c-5c55-487f-8de6-200bc0be3e06-kube-api-access-nq4hg\") pod \"node-ca-r4h2b\" (UID: \"416afa6c-5c55-487f-8de6-200bc0be3e06\") " pod="openshift-image-registry/node-ca-r4h2b"
Sep 14 07:48:01.967326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-etc-kubernetes\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.967326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/416afa6c-5c55-487f-8de6-200bc0be3e06-host\") pod \"node-ca-r4h2b\" (UID: \"416afa6c-5c55-487f-8de6-200bc0be3e06\") " pod="openshift-image-registry/node-ca-r4h2b"
Sep 14 07:48:01.967326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-kubernetes\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.967326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-sysctl-d\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.967326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/416afa6c-5c55-487f-8de6-200bc0be3e06-serviceca\") pod \"node-ca-r4h2b\" (UID: \"416afa6c-5c55-487f-8de6-200bc0be3e06\") " pod="openshift-image-registry/node-ca-r4h2b"
Sep 14 07:48:01.967326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967270    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-lib-modules\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.967326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967314    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-run-k8s-cni-cncf-io\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967336    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-etc-kubernetes\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-run-netns\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967354    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-multus-socket-dir-parent\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967371    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-run\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967399    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-run-netns\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967403    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5zfx\" (UniqueName: \"kubernetes.io/projected/261155e3-6b52-449b-94ca-649f7210ccd9-kube-api-access-q5zfx\") pod \"node-resolver-ts5z9\" (UID: \"261155e3-6b52-449b-94ca-649f7210ccd9\") " pod="openshift-dns/node-resolver-ts5z9"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967435    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-var-lib-cni-multus\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967462    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7v8h\" (UniqueName: \"kubernetes.io/projected/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-kube-api-access-l7v8h\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967488    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/261155e3-6b52-449b-94ca-649f7210ccd9-hosts-file\") pod \"node-resolver-ts5z9\" (UID: \"261155e3-6b52-449b-94ca-649f7210ccd9\") " pod="openshift-dns/node-resolver-ts5z9"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/261155e3-6b52-449b-94ca-649f7210ccd9-tmp-dir\") pod \"node-resolver-ts5z9\" (UID: \"261155e3-6b52-449b-94ca-649f7210ccd9\") " pod="openshift-dns/node-resolver-ts5z9"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967270    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-systemd\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-sysconfig\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967376    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/416afa6c-5c55-487f-8de6-200bc0be3e06-host\") pod \"node-ca-r4h2b\" (UID: \"416afa6c-5c55-487f-8de6-200bc0be3e06\") " pod="openshift-image-registry/node-ca-r4h2b"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967586    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bee2d6cd-aca8-4008-be09-34cd9916d966-kubelet-config\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967614    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-var-lib-cni-bin\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967638    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-var-lib-kubelet\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967667    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-tuned\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.967837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-system-cni-dir\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967747    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-var-lib-kubelet\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-run-multus-certs\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-multus-cni-dir\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-hostroot\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967851    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-sysconfig\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967537    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-sysctl-d\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967915    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-var-lib-kubelet\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967958    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-run\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967992    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/261155e3-6b52-449b-94ca-649f7210ccd9-tmp-dir\") pod \"node-resolver-ts5z9\" (UID: \"261155e3-6b52-449b-94ca-649f7210ccd9\") " pod="openshift-dns/node-resolver-ts5z9"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968026    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bee2d6cd-aca8-4008-be09-34cd9916d966-kubelet-config\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-host\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-cnibin\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-sys\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvs5\" (UniqueName: \"kubernetes.io/projected/1a1a3710-8da0-4352-9225-64e4ebac48fd-kube-api-access-zjvs5\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968122    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/416afa6c-5c55-487f-8de6-200bc0be3e06-serviceca\") pod \"node-ca-r4h2b\" (UID: \"416afa6c-5c55-487f-8de6-200bc0be3e06\") " pod="openshift-image-registry/node-ca-r4h2b"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967465    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-kubernetes\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968169    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-cnibin\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.968749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-multus-conf-dir\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968222    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-var-lib-kubelet\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-multus-daemon-config\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/261155e3-6b52-449b-94ca-649f7210ccd9-hosts-file\") pod \"node-resolver-ts5z9\" (UID: \"261155e3-6b52-449b-94ca-649f7210ccd9\") " pod="openshift-dns/node-resolver-ts5z9"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968264    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-var-lib-cni-multus\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968273    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-sysctl-conf\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968297    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-multus-conf-dir\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a1a3710-8da0-4352-9225-64e4ebac48fd-tmp\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bee2d6cd-aca8-4008-be09-34cd9916d966-dbus\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968352    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-sys\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968400    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-sysctl-conf\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968410    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-system-cni-dir\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968423    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-cni-binary-copy\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968449    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-run-multus-certs\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968452    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-modprobe-d\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:01.968479    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:01.968537    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret podName:bee2d6cd-aca8-4008-be09-34cd9916d966 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:02.468520412 +0000 UTC m=+3.097408034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret") pod "global-pull-secret-syncer-zwmpr" (UID: "bee2d6cd-aca8-4008-be09-34cd9916d966") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:01.969650 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968588    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-modprobe-d\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.970551 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968667    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-multus-cni-dir\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.970551 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.967802    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-host-var-lib-cni-bin\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.970551 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968752    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a1a3710-8da0-4352-9225-64e4ebac48fd-host\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.970551 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968764    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-hostroot\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.970551 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.968822    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bee2d6cd-aca8-4008-be09-34cd9916d966-dbus\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:01.970551 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.969040    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-multus-daemon-config\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.970551 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.969489    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-cni-binary-copy\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.970937 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.970777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1a1a3710-8da0-4352-9225-64e4ebac48fd-etc-tuned\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.971038 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.971017    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a1a3710-8da0-4352-9225-64e4ebac48fd-tmp\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.980969 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.980428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5zfx\" (UniqueName: \"kubernetes.io/projected/261155e3-6b52-449b-94ca-649f7210ccd9-kube-api-access-q5zfx\") pod \"node-resolver-ts5z9\" (UID: \"261155e3-6b52-449b-94ca-649f7210ccd9\") " pod="openshift-dns/node-resolver-ts5z9"
Sep 14 07:48:01.980969 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.980440    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7v8h\" (UniqueName: \"kubernetes.io/projected/ef0aa66b-f097-4d15-b247-93ec7a1bdb3c-kube-api-access-l7v8h\") pod \"multus-7z55c\" (UID: \"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c\") " pod="openshift-multus/multus-7z55c"
Sep 14 07:48:01.980969 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.980872    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjvs5\" (UniqueName: \"kubernetes.io/projected/1a1a3710-8da0-4352-9225-64e4ebac48fd-kube-api-access-zjvs5\") pod \"tuned-zwg2q\" (UID: \"1a1a3710-8da0-4352-9225-64e4ebac48fd\") " pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:01.980969 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:01.980933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq4hg\" (UniqueName: \"kubernetes.io/projected/416afa6c-5c55-487f-8de6-200bc0be3e06-kube-api-access-nq4hg\") pod \"node-ca-r4h2b\" (UID: \"416afa6c-5c55-487f-8de6-200bc0be3e06\") " pod="openshift-image-registry/node-ca-r4h2b"
Sep 14 07:48:02.051307 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.051259    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ffvbt"
Sep 14 07:48:02.065170 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.065135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v22pk"
Sep 14 07:48:02.077598 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.077573    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wszct"
Sep 14 07:48:02.086633 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.086604    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:02.094702 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.094679    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c"
Sep 14 07:48:02.104518 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.104493    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zwg2q"
Sep 14 07:48:02.113544 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.113511    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ts5z9"
Sep 14 07:48:02.124488 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.124461    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r4h2b"
Sep 14 07:48:02.131549 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.131522    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7z55c"
Sep 14 07:48:02.134187 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.134166    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:48:02.371975 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.371865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:02.372166 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:02.372057    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:02.372166 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:02.372142    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs podName:c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:03.372120422 +0000 UTC m=+4.001008033 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs") pod "network-metrics-daemon-46qgp" (UID: "c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:02.472834 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.472778    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrp8\" (UniqueName: \"kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8\") pod \"network-check-target-vpz6b\" (UID: \"e47647c7-20f1-440e-83da-65828c7f4605\") " pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:02.472834 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.472847    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:02.473095 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:02.472983    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:02.473095 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:02.473019    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:02.473095 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:02.473033    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8hrp8 for pod openshift-network-diagnostics/network-check-target-vpz6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:02.473095 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:02.472983    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:02.473299 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:02.473118    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8 podName:e47647c7-20f1-440e-83da-65828c7f4605 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:03.47309934 +0000 UTC m=+4.101986963 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8hrp8" (UniqueName: "kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8") pod "network-check-target-vpz6b" (UID: "e47647c7-20f1-440e-83da-65828c7f4605") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:02.473299 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:02.473141    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret podName:bee2d6cd-aca8-4008-be09-34cd9916d966 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:03.473127499 +0000 UTC m=+4.102015105 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret") pod "global-pull-secret-syncer-zwmpr" (UID: "bee2d6cd-aca8-4008-be09-34cd9916d966") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:02.796341 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.796248    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 07:43:02 +0000 UTC" deadline="2028-04-17 16:46:12.43620177 +0000 UTC"
Sep 14 07:48:02.796341 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.796290    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13952h58m9.639921403s"
Sep 14 07:48:02.880127 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.880086    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:02.880302 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:02.880133    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:02.880302 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:02.880234    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:02.880419 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:02.880373    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:02.942602 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:02.942566    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61f94071_3e09_4079_abb1_bbfa74ae8453.slice/crio-9f45427433402d9994d92e0d2c771f45e78634f21dd17dca06d74101a26531df WatchSource:0}: Error finding container 9f45427433402d9994d92e0d2c771f45e78634f21dd17dca06d74101a26531df: Status 404 returned error can't find the container with id 9f45427433402d9994d92e0d2c771f45e78634f21dd17dca06d74101a26531df
Sep 14 07:48:02.943621 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:02.943595    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef0aa66b_f097_4d15_b247_93ec7a1bdb3c.slice/crio-12249a22c2a2ba03a5fc6f0a21af6b2e4655c9a17d629084a8654ea98d20c178 WatchSource:0}: Error finding container 12249a22c2a2ba03a5fc6f0a21af6b2e4655c9a17d629084a8654ea98d20c178: Status 404 returned error can't find the container with id 12249a22c2a2ba03a5fc6f0a21af6b2e4655c9a17d629084a8654ea98d20c178
Sep 14 07:48:02.945327 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:02.944978    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f00ee5_9070_420e_9fe1_3e5e0482610e.slice/crio-8e5352a1746c03d09613b82ac342506e7c0eebeff6a49cc86fcbbf6baba51193 WatchSource:0}: Error finding container 8e5352a1746c03d09613b82ac342506e7c0eebeff6a49cc86fcbbf6baba51193: Status 404 returned error can't find the container with id 8e5352a1746c03d09613b82ac342506e7c0eebeff6a49cc86fcbbf6baba51193
Sep 14 07:48:02.946015 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:02.945889    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0fb32a4_26e3_4097_9577_6c7057da8697.slice/crio-2b2ab2ceadb20cae5a6e58002a2b760ad306af7a199091ce2a444f5349378933 WatchSource:0}: Error finding container 2b2ab2ceadb20cae5a6e58002a2b760ad306af7a199091ce2a444f5349378933: Status 404 returned error can't find the container with id 2b2ab2ceadb20cae5a6e58002a2b760ad306af7a199091ce2a444f5349378933
Sep 14 07:48:02.949649 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:02.948764    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a1a3710_8da0_4352_9225_64e4ebac48fd.slice/crio-c831b2219e781a79218f73f1d94f61fbab547259bd6cc3699f52f2a896930c13 WatchSource:0}: Error finding container c831b2219e781a79218f73f1d94f61fbab547259bd6cc3699f52f2a896930c13: Status 404 returned error can't find the container with id c831b2219e781a79218f73f1d94f61fbab547259bd6cc3699f52f2a896930c13
Sep 14 07:48:02.952414 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:02.952390    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17454c80_e6af_4f63_bc8c_84a6f1cb536b.slice/crio-cfcfc0f411701022076eae9896c87d342c74d341eb599dd48c206fefd3a948d8 WatchSource:0}: Error finding container cfcfc0f411701022076eae9896c87d342c74d341eb599dd48c206fefd3a948d8: Status 404 returned error can't find the container with id cfcfc0f411701022076eae9896c87d342c74d341eb599dd48c206fefd3a948d8
Sep 14 07:48:02.953823 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:02.953794    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d1ffd7_116f_4af9_bf09_5e3df09c4155.slice/crio-43fa420c108264d0880c86bf6000a61ab1282d18393ff241e76daf2a34239378 WatchSource:0}: Error finding container 43fa420c108264d0880c86bf6000a61ab1282d18393ff241e76daf2a34239378: Status 404 returned error can't find the container with id 43fa420c108264d0880c86bf6000a61ab1282d18393ff241e76daf2a34239378
Sep 14 07:48:02.954597 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:02.954574    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416afa6c_5c55_487f_8de6_200bc0be3e06.slice/crio-a36058bc360d71352d2418e47dace33b7fb4255c57464f1ba1368670103fa031 WatchSource:0}: Error finding container a36058bc360d71352d2418e47dace33b7fb4255c57464f1ba1368670103fa031: Status 404 returned error can't find the container with id a36058bc360d71352d2418e47dace33b7fb4255c57464f1ba1368670103fa031
Sep 14 07:48:02.956018 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:02.955999    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261155e3_6b52_449b_94ca_649f7210ccd9.slice/crio-2a3dc91fa601a1b121f11a6c4d50c9381af4ec7941d808b64927452f18704f5c WatchSource:0}: Error finding container 2a3dc91fa601a1b121f11a6c4d50c9381af4ec7941d808b64927452f18704f5c: Status 404 returned error can't find the container with id 2a3dc91fa601a1b121f11a6c4d50c9381af4ec7941d808b64927452f18704f5c
Sep 14 07:48:03.056103 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.055849    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:48:03.381424 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.381335    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:03.381586 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:03.381498    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:03.381586 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:03.381583    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs podName:c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:05.381559825 +0000 UTC m=+6.010447432 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs") pod "network-metrics-daemon-46qgp" (UID: "c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:03.482449 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.482399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrp8\" (UniqueName: \"kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8\") pod \"network-check-target-vpz6b\" (UID: \"e47647c7-20f1-440e-83da-65828c7f4605\") " pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:03.482449 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.482452    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:03.482693 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:03.482597    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:03.482693 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:03.482662    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret podName:bee2d6cd-aca8-4008-be09-34cd9916d966 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:05.482643065 +0000 UTC m=+6.111530690 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret") pod "global-pull-secret-syncer-zwmpr" (UID: "bee2d6cd-aca8-4008-be09-34cd9916d966") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:03.483137 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:03.483114    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:03.483248 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:03.483142    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:03.483248 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:03.483156    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8hrp8 for pod openshift-network-diagnostics/network-check-target-vpz6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:03.483248 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:03.483209    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8 podName:e47647c7-20f1-440e-83da-65828c7f4605 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:05.48319272 +0000 UTC m=+6.112080330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8hrp8" (UniqueName: "kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8") pod "network-check-target-vpz6b" (UID: "e47647c7-20f1-440e-83da-65828c7f4605") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:03.797326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.797198    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 07:43:02 +0000 UTC" deadline="2028-03-04 14:52:09.269832773 +0000 UTC"
Sep 14 07:48:03.797326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.797270    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12895h4m5.472566907s"
Sep 14 07:48:03.881165 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.881107    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:03.881329 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:03.881256    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:03.901762 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.901703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wszct" event={"ID":"a8d1ffd7-116f-4af9-bf09-5e3df09c4155","Type":"ContainerStarted","Data":"43fa420c108264d0880c86bf6000a61ab1282d18393ff241e76daf2a34239378"}
Sep 14 07:48:03.904902 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.904866    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ts5z9" event={"ID":"261155e3-6b52-449b-94ca-649f7210ccd9","Type":"ContainerStarted","Data":"2a3dc91fa601a1b121f11a6c4d50c9381af4ec7941d808b64927452f18704f5c"}
Sep 14 07:48:03.923134 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.923088    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r4h2b" event={"ID":"416afa6c-5c55-487f-8de6-200bc0be3e06","Type":"ContainerStarted","Data":"a36058bc360d71352d2418e47dace33b7fb4255c57464f1ba1368670103fa031"}
Sep 14 07:48:03.925794 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.925763    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c" event={"ID":"17454c80-e6af-4f63-bc8c-84a6f1cb536b","Type":"ContainerStarted","Data":"cfcfc0f411701022076eae9896c87d342c74d341eb599dd48c206fefd3a948d8"}
Sep 14 07:48:03.933077 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.933045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zwg2q" event={"ID":"1a1a3710-8da0-4352-9225-64e4ebac48fd","Type":"ContainerStarted","Data":"c831b2219e781a79218f73f1d94f61fbab547259bd6cc3699f52f2a896930c13"}
Sep 14 07:48:03.948459 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.948341    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f" event={"ID":"c0fb32a4-26e3-4097-9577-6c7057da8697","Type":"ContainerStarted","Data":"2b2ab2ceadb20cae5a6e58002a2b760ad306af7a199091ce2a444f5349378933"}
Sep 14 07:48:03.973701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.973664    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ffvbt" event={"ID":"61f94071-3e09-4079-abb1-bbfa74ae8453","Type":"ContainerStarted","Data":"9f45427433402d9994d92e0d2c771f45e78634f21dd17dca06d74101a26531df"}
Sep 14 07:48:03.984227 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.984181    2563 generic.go:356] "Generic (PLEG): container finished" podID="ecb1680172f83e440b8fb7199dfde94a" containerID="0df884118373955707bf4e732fccfdf9ab3c78eb8517a097582571bb3423f721" exitCode=0
Sep 14 07:48:03.984543 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.984513    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal" event={"ID":"ecb1680172f83e440b8fb7199dfde94a","Type":"ContainerDied","Data":"0df884118373955707bf4e732fccfdf9ab3c78eb8517a097582571bb3423f721"}
Sep 14 07:48:03.994230 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:03.994195    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7z55c" event={"ID":"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c","Type":"ContainerStarted","Data":"12249a22c2a2ba03a5fc6f0a21af6b2e4655c9a17d629084a8654ea98d20c178"}
Sep 14 07:48:04.007851 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:04.007818    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pk" event={"ID":"d9f00ee5-9070-420e-9fe1-3e5e0482610e","Type":"ContainerStarted","Data":"8e5352a1746c03d09613b82ac342506e7c0eebeff6a49cc86fcbbf6baba51193"}
Sep 14 07:48:04.037975 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:04.037934    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-250.ec2.internal" event={"ID":"520462c720d404adb820140db50e2930","Type":"ContainerStarted","Data":"07d37633b9b7948a19cefd730176a49ec430628bc704adda9d28952a441fb6fb"}
Sep 14 07:48:04.054804 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:04.054740    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-250.ec2.internal" podStartSLOduration=3.05470093 podStartE2EDuration="3.05470093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:04.053835365 +0000 UTC m=+4.682722997" watchObservedRunningTime="2026-09-14 07:48:04.05470093 +0000 UTC m=+4.683588561"
Sep 14 07:48:04.880270 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:04.880234    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:04.881117 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:04.880379    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:04.881117 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:04.880813    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:04.881117 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:04.880936    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:05.046439 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:05.046336    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal" event={"ID":"ecb1680172f83e440b8fb7199dfde94a","Type":"ContainerStarted","Data":"2dbf733f9c25a52b4b33c98deccda13c163a146f88742283e77826a53833fe90"}
Sep 14 07:48:05.400500 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:05.400407    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:05.400680 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:05.400640    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:05.400680 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:05.400727    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs podName:c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:09.400706343 +0000 UTC m=+10.029593977 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs") pod "network-metrics-daemon-46qgp" (UID: "c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:05.504149 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:05.502293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrp8\" (UniqueName: \"kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8\") pod \"network-check-target-vpz6b\" (UID: \"e47647c7-20f1-440e-83da-65828c7f4605\") " pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:05.504149 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:05.502347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:05.504149 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:05.502481    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:05.504149 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:05.502545    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret podName:bee2d6cd-aca8-4008-be09-34cd9916d966 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:09.502526017 +0000 UTC m=+10.131413632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret") pod "global-pull-secret-syncer-zwmpr" (UID: "bee2d6cd-aca8-4008-be09-34cd9916d966") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:05.504149 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:05.503003    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:05.504149 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:05.503025    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:05.504149 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:05.503039    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8hrp8 for pod openshift-network-diagnostics/network-check-target-vpz6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:05.504149 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:05.503094    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8 podName:e47647c7-20f1-440e-83da-65828c7f4605 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:09.503075909 +0000 UTC m=+10.131963523 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8hrp8" (UniqueName: "kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8") pod "network-check-target-vpz6b" (UID: "e47647c7-20f1-440e-83da-65828c7f4605") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:05.879948 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:05.879865    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:05.880134 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:05.880004    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:06.879833 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:06.879796    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:06.880296 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:06.879926    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:06.880296 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:06.880023    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:06.880296 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:06.880110    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:07.883783 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:07.883753    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:07.884266 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:07.883875    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:08.880422 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:08.880098    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:08.880422 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:08.880236    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:08.880422 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:08.880294    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:08.880728 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:08.880430    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:09.436027 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:09.435990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:09.436481 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:09.436180    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:09.436481 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:09.436252    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs podName:c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:17.436230608 +0000 UTC m=+18.065118236 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs") pod "network-metrics-daemon-46qgp" (UID: "c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:09.538018 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:09.537280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrp8\" (UniqueName: \"kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8\") pod \"network-check-target-vpz6b\" (UID: \"e47647c7-20f1-440e-83da-65828c7f4605\") " pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:09.538018 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:09.537329    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:09.538018 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:09.537444    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:09.538018 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:09.537497    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret podName:bee2d6cd-aca8-4008-be09-34cd9916d966 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:17.537483098 +0000 UTC m=+18.166370702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret") pod "global-pull-secret-syncer-zwmpr" (UID: "bee2d6cd-aca8-4008-be09-34cd9916d966") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:09.538018 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:09.537903    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:09.538018 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:09.537927    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:09.538018 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:09.537941    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8hrp8 for pod openshift-network-diagnostics/network-check-target-vpz6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:09.538018 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:09.537979    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8 podName:e47647c7-20f1-440e-83da-65828c7f4605 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:17.537967667 +0000 UTC m=+18.166855271 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8hrp8" (UniqueName: "kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8") pod "network-check-target-vpz6b" (UID: "e47647c7-20f1-440e-83da-65828c7f4605") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:09.883622 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:09.883539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:09.883786 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:09.883683    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:10.879940 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:10.879340    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:10.879940 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:10.879378    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:10.879940 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:10.879465    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:10.879940 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:10.879903    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:11.879755 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:11.879720    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:11.879915 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:11.879838    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:12.880370 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:12.880332    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:12.880819 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:12.880332    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:12.880819 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:12.880484    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:12.880819 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:12.880545    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:13.879949 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:13.879914    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:13.880138 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:13.880056    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:14.879773 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:14.879723    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:14.879773 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:14.879770    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:14.880305 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:14.879861    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:14.880305 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:14.880024    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:15.879656 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:15.879615    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:15.879828 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:15.879762    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:16.879902 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:16.879868    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:16.880322 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:16.879993    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:16.880322 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:16.880057    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:16.880322 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:16.880190    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:17.495800 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:17.495764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:17.495963 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:17.495871    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:17.495963 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:17.495921    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs podName:c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:33.49590668 +0000 UTC m=+34.124794285 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs") pod "network-metrics-daemon-46qgp" (UID: "c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:17.596433 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:17.596402    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrp8\" (UniqueName: \"kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8\") pod \"network-check-target-vpz6b\" (UID: \"e47647c7-20f1-440e-83da-65828c7f4605\") " pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:17.596433 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:17.596436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:17.596608 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:17.596528    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:17.596608 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:17.596549    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:17.596608 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:17.596573    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:17.596608 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:17.596577    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret podName:bee2d6cd-aca8-4008-be09-34cd9916d966 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:33.596562912 +0000 UTC m=+34.225450518 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret") pod "global-pull-secret-syncer-zwmpr" (UID: "bee2d6cd-aca8-4008-be09-34cd9916d966") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:17.596608 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:17.596585    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8hrp8 for pod openshift-network-diagnostics/network-check-target-vpz6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:17.596798 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:17.596625    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8 podName:e47647c7-20f1-440e-83da-65828c7f4605 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:33.596613186 +0000 UTC m=+34.225500796 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8hrp8" (UniqueName: "kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8") pod "network-check-target-vpz6b" (UID: "e47647c7-20f1-440e-83da-65828c7f4605") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:17.879565 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:17.879528    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:17.879772 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:17.879696    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:18.879657 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:18.879608    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:18.880180 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:18.879633    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:18.880180 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:18.879755    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:18.880180 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:18.879828    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:19.880912 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:19.880867    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:19.881364 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:19.880991    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:20.879827 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:20.879796    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:20.879989 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:20.879796    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:20.879989 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:20.879922    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:20.880105 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:20.879985    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:21.098245 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:21.096885    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zwg2q" event={"ID":"1a1a3710-8da0-4352-9225-64e4ebac48fd","Type":"ContainerStarted","Data":"1c1ee1bd176110bf2781d4b35f0e4a3db90887ac1b9290f9794f677d8cc40dc6"}
Sep 14 07:48:21.103372 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:21.103310    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ffvbt" event={"ID":"61f94071-3e09-4079-abb1-bbfa74ae8453","Type":"ContainerStarted","Data":"ecf6b648131e402df290013b247509334ceda5faa712fc20cc60377659f0dad8"}
Sep 14 07:48:21.124590 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:21.124544    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-250.ec2.internal" podStartSLOduration=20.12452861 podStartE2EDuration="20.12452861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:05.062998717 +0000 UTC m=+5.691886351" watchObservedRunningTime="2026-09-14 07:48:21.12452861 +0000 UTC m=+21.753416237"
Sep 14 07:48:21.124807 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:21.124784    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zwg2q" podStartSLOduration=3.191030778 podStartE2EDuration="21.12477639s" totalImagesPullingTime="17.933745612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:21.122472763 +0000 UTC m=+21.751360391" watchObservedRunningTime="2026-09-14 07:48:21.12477639 +0000 UTC m=+21.753664018"
Sep 14 07:48:21.147187 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:21.147131    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7z55c" podStartSLOduration=3.148703792 podStartE2EDuration="21.147107634s" totalImagesPullingTime="17.998403842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:21.146055945 +0000 UTC m=+21.774943589" watchObservedRunningTime="2026-09-14 07:48:21.147107634 +0000 UTC m=+21.775995264"
Sep 14 07:48:21.164624 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:21.164474    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ffvbt" podStartSLOduration=10.348708655 podStartE2EDuration="21.164458501s" totalImagesPullingTime="10.815749846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:21.164250741 +0000 UTC m=+21.793138469" watchObservedRunningTime="2026-09-14 07:48:21.164458501 +0000 UTC m=+21.793346127"
Sep 14 07:48:21.880176 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:21.880134    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:21.880354 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:21.880280    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:22.110602 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.110568    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ts5z9" event={"ID":"261155e3-6b52-449b-94ca-649f7210ccd9","Type":"ContainerStarted","Data":"7bd682f35fe4a0704f7c26e845acb0428b42370147fb6d76c020b2d47895a47e"}
Sep 14 07:48:22.112072 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.112042    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r4h2b" event={"ID":"416afa6c-5c55-487f-8de6-200bc0be3e06","Type":"ContainerStarted","Data":"4ae19ec3ef43b690ae1a3066c9bb27eff1878038ae1db0a12d9c4849cacb816d"}
Sep 14 07:48:22.113473 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.113454    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c" event={"ID":"17454c80-e6af-4f63-bc8c-84a6f1cb536b","Type":"ContainerStarted","Data":"142b1e5a8d47e417c638d701c9d37b0eee15174882e72003fb03f5636ce5051f"}
Sep 14 07:48:22.116350 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.116330    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f" event={"ID":"c0fb32a4-26e3-4097-9577-6c7057da8697","Type":"ContainerStarted","Data":"37a8a1ad37d076a1503b9b266918d02d21fdb60640c6220c16ed89cadb78a0c5"}
Sep 14 07:48:22.116457 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.116387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f" event={"ID":"c0fb32a4-26e3-4097-9577-6c7057da8697","Type":"ContainerStarted","Data":"34434e1612eeca6c4a00154fd1e8aeb404a31e92aa2311b3d1b00a4220f13c4a"}
Sep 14 07:48:22.116457 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.116418    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f" event={"ID":"c0fb32a4-26e3-4097-9577-6c7057da8697","Type":"ContainerStarted","Data":"ccf0fbfa203bcb3683f1d563defb57c0290d4543862ea4ef7aef29ffd7c46a07"}
Sep 14 07:48:22.116457 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.116454    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f" event={"ID":"c0fb32a4-26e3-4097-9577-6c7057da8697","Type":"ContainerStarted","Data":"7b68380ed2fb40628ed10a2ae241843bb5bd9310f90bce9b56326042e0a729ab"}
Sep 14 07:48:22.116576 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.116483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f" event={"ID":"c0fb32a4-26e3-4097-9577-6c7057da8697","Type":"ContainerStarted","Data":"574add6f76e17bd0a347a6e02542a3a8061daab004bd1edc80da574c6b0dcee3"}
Sep 14 07:48:22.116576 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.116510    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f" event={"ID":"c0fb32a4-26e3-4097-9577-6c7057da8697","Type":"ContainerStarted","Data":"65c635bf5b6bcabfc678640e713ce905802d759cfd9e14f035a8aa5ba2a1ba3f"}
Sep 14 07:48:22.117755 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.117734    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7z55c" event={"ID":"ef0aa66b-f097-4d15-b247-93ec7a1bdb3c","Type":"ContainerStarted","Data":"156ea0879c631c67f2e1ca764328b419a70e019836d7aa612fd52caeaf8967b0"}
Sep 14 07:48:22.119031 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.119008    2563 generic.go:356] "Generic (PLEG): container finished" podID="d9f00ee5-9070-420e-9fe1-3e5e0482610e" containerID="b4fc7f5cfab8213180672765a688abbf92ef520f610c0f5f7380e337f9ef475b" exitCode=0
Sep 14 07:48:22.119117 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.119077    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pk" event={"ID":"d9f00ee5-9070-420e-9fe1-3e5e0482610e","Type":"ContainerDied","Data":"b4fc7f5cfab8213180672765a688abbf92ef520f610c0f5f7380e337f9ef475b"}
Sep 14 07:48:22.127505 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.127466    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ts5z9" podStartSLOduration=4.205756728 podStartE2EDuration="22.127454732s" totalImagesPullingTime="17.921698004s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:22.127026007 +0000 UTC m=+22.755913653" watchObservedRunningTime="2026-09-14 07:48:22.127454732 +0000 UTC m=+22.756342358"
Sep 14 07:48:22.164756 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.164719    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-r4h2b" podStartSLOduration=4.239491307 podStartE2EDuration="22.164706806s" totalImagesPullingTime="17.925215499s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:22.164375686 +0000 UTC m=+22.793263312" watchObservedRunningTime="2026-09-14 07:48:22.164706806 +0000 UTC m=+22.793594433"
Sep 14 07:48:22.385940 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.385915    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 14 07:48:22.825035 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.824863    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-14T07:48:22.38593597Z","UUID":"9c6bb2c8-6569-4359-ae3d-f9e7f91a005c","Handler":null,"Name":"","Endpoint":""}
Sep 14 07:48:22.826432 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.826408    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 14 07:48:22.826576 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.826444    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 14 07:48:22.879378 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.879348    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:22.879569 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:22.879479    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:22.879569 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:22.879533    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:22.879755 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:22.879670    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:23.123953 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:23.123914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wszct" event={"ID":"a8d1ffd7-116f-4af9-bf09-5e3df09c4155","Type":"ContainerStarted","Data":"d8add15e9cf2d013ca43b584a4bf87254a5adeafe26b1900927cf4dc86628a5a"}
Sep 14 07:48:23.126295 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:23.126266    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c" event={"ID":"17454c80-e6af-4f63-bc8c-84a6f1cb536b","Type":"ContainerStarted","Data":"7da8bea924acd8d565d57c6e479f7c2d896f20b64cf356754a9b1d69691525c4"}
Sep 14 07:48:23.879507 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:23.879470    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:23.879717 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:23.879610    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:24.130277 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:24.130195    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c" event={"ID":"17454c80-e6af-4f63-bc8c-84a6f1cb536b","Type":"ContainerStarted","Data":"aad8b7201fbe496f301164596e7ebcb3439c8088ba36978b97a7fe98c826c5d6"}
Sep 14 07:48:24.133608 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:24.133588    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f" event={"ID":"c0fb32a4-26e3-4097-9577-6c7057da8697","Type":"ContainerStarted","Data":"f1d1d33842ec7c8dee3c856b136e142cca2ce1e2b53f787749d4f5013ade469f"}
Sep 14 07:48:24.150007 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:24.149966    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wszct" podStartSLOduration=6.223635264 podStartE2EDuration="24.149953665s" totalImagesPullingTime="17.926318401s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:23.149722523 +0000 UTC m=+23.778610172" watchObservedRunningTime="2026-09-14 07:48:24.149953665 +0000 UTC m=+24.778841292"
Sep 14 07:48:24.880239 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:24.880202    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:24.880429 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:24.880315    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:24.880429 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:24.880408    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:24.880665 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:24.880612    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:25.304671 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:25.304392    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ffvbt"
Sep 14 07:48:25.305359 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:25.305335    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ffvbt"
Sep 14 07:48:25.322845 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:25.322799    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wjb9c" podStartSLOduration=5.05914665 podStartE2EDuration="25.322782759s" totalImagesPullingTime="20.263636109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:24.149559801 +0000 UTC m=+24.778447437" watchObservedRunningTime="2026-09-14 07:48:25.322782759 +0000 UTC m=+25.951670386"
Sep 14 07:48:25.880147 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:25.880108    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:25.880326 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:25.880249    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:26.879734 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:26.879698    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:26.880065 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:26.879706    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:26.880065 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:26.879801    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:26.880065 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:26.879877    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:27.144893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:27.144866    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f" event={"ID":"c0fb32a4-26e3-4097-9577-6c7057da8697","Type":"ContainerStarted","Data":"4d5590e00cf32281f3d352fb8719b9f69bb61819f62b9002fb4e54526e9a91e9"}
Sep 14 07:48:27.145171 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:27.145146    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:27.145271 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:27.145222    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:27.146806 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:27.146786    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pk" event={"ID":"d9f00ee5-9070-420e-9fe1-3e5e0482610e","Type":"ContainerStarted","Data":"725963b44bc0e7e6d18024201617a5240d7f268ee985f22d0fbee57de36bd0c4"}
Sep 14 07:48:27.160552 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:27.160532    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:27.228976 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:27.228933    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f" podStartSLOduration=9.228928099 podStartE2EDuration="27.22891895s" totalImagesPullingTime="17.999990851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:27.195098413 +0000 UTC m=+27.823986039" watchObservedRunningTime="2026-09-14 07:48:27.22891895 +0000 UTC m=+27.857806577"
Sep 14 07:48:27.879700 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:27.879479    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:27.879867 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:27.879790    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:28.151043 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:28.150975    2563 generic.go:356] "Generic (PLEG): container finished" podID="d9f00ee5-9070-420e-9fe1-3e5e0482610e" containerID="725963b44bc0e7e6d18024201617a5240d7f268ee985f22d0fbee57de36bd0c4" exitCode=0
Sep 14 07:48:28.151161 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:28.151044    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pk" event={"ID":"d9f00ee5-9070-420e-9fe1-3e5e0482610e","Type":"ContainerDied","Data":"725963b44bc0e7e6d18024201617a5240d7f268ee985f22d0fbee57de36bd0c4"}
Sep 14 07:48:28.151578 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:28.151505    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:28.166720 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:28.166701    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:48:28.682684 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:28.682589    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zwmpr"]
Sep 14 07:48:28.682846 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:28.682831    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:28.682986 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:28.682949    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:28.685725 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:28.685697    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vpz6b"]
Sep 14 07:48:28.685903 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:28.685818    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:28.685903 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:28.685903    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:28.687276 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:28.687250    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-46qgp"]
Sep 14 07:48:28.687458 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:28.687357    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:28.687512 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:28.687454    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:29.403697 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:29.403672    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:29.485236 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:29.485201    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:29.880940 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:29.880908    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:29.881126 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:29.881006    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:29.881126 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:29.881083    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:29.881233 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:29.881177    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:30.157499 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:30.157464    2563 generic.go:356] "Generic (PLEG): container finished" podID="d9f00ee5-9070-420e-9fe1-3e5e0482610e" containerID="d1ce0692abd403ebe9900c3daba0be3208bf8af77ec3d6e4b112289d3ecf0fc0" exitCode=0
Sep 14 07:48:30.157626 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:30.157536    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pk" event={"ID":"d9f00ee5-9070-420e-9fe1-3e5e0482610e","Type":"ContainerDied","Data":"d1ce0692abd403ebe9900c3daba0be3208bf8af77ec3d6e4b112289d3ecf0fc0"}
Sep 14 07:48:30.879983 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:30.879804    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:30.880325 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:30.880056    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:31.162599 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:31.162566    2563 generic.go:356] "Generic (PLEG): container finished" podID="d9f00ee5-9070-420e-9fe1-3e5e0482610e" containerID="57e2f42f2aca723f3911587f2dba702f2a1420dd3a9ba80a3d698bde283cc3ef" exitCode=0
Sep 14 07:48:31.162599 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:31.162610    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pk" event={"ID":"d9f00ee5-9070-420e-9fe1-3e5e0482610e","Type":"ContainerDied","Data":"57e2f42f2aca723f3911587f2dba702f2a1420dd3a9ba80a3d698bde283cc3ef"}
Sep 14 07:48:31.516207 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:31.516130    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ffvbt"
Sep 14 07:48:31.516356 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:31.516250    2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 07:48:31.516836 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:31.516816    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ffvbt"
Sep 14 07:48:31.593875 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:31.593837    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:31.880243 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:31.880212    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:31.880713 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:31.880212    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:31.880713 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:31.880351    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46qgp" podUID="c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257"
Sep 14 07:48:31.880713 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:31.880407    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zwmpr" podUID="bee2d6cd-aca8-4008-be09-34cd9916d966"
Sep 14 07:48:32.879842 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:32.879806    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:32.880121 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:32.879927    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpz6b" podUID="e47647c7-20f1-440e-83da-65828c7f4605"
Sep 14 07:48:33.022939 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.022910    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:33.120491 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.120425    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:33.513063 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.513034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:33.513252 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:33.513226    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:33.513345 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:33.513332    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs podName:c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:05.513304412 +0000 UTC m=+66.142192020 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs") pod "network-metrics-daemon-46qgp" (UID: "c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:33.613477 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.613444    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:33.613679 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.613549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrp8\" (UniqueName: \"kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8\") pod \"network-check-target-vpz6b\" (UID: \"e47647c7-20f1-440e-83da-65828c7f4605\") " pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:33.613679 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:33.613617    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:33.613799 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:33.613687    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:33.613799 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:33.613704    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:33.613799 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:33.613718    2563 projected.go:196] Error preparing data for projected volume kube-api-access-8hrp8 for pod openshift-network-diagnostics/network-check-target-vpz6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:33.613799 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:33.613706    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret podName:bee2d6cd-aca8-4008-be09-34cd9916d966 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:05.613683283 +0000 UTC m=+66.242570891 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret") pod "global-pull-secret-syncer-zwmpr" (UID: "bee2d6cd-aca8-4008-be09-34cd9916d966") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:33.613799 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:33.613782    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8 podName:e47647c7-20f1-440e-83da-65828c7f4605 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:05.613759895 +0000 UTC m=+66.242647509 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8hrp8" (UniqueName: "kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8") pod "network-check-target-vpz6b" (UID: "e47647c7-20f1-440e-83da-65828c7f4605") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:33.641880 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.641855    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-250.ec2.internal" event="NodeReady"
Sep 14 07:48:33.642018 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.642003    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 14 07:48:33.693785 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.693706    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mk8vk"]
Sep 14 07:48:33.701440 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.701409    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s6ld9"]
Sep 14 07:48:33.701585 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.701549    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:33.703824 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.703776    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7986b\""
Sep 14 07:48:33.703945 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.703927    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 14 07:48:33.704043 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.704028    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 14 07:48:33.705363 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.705021    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s6ld9"
Sep 14 07:48:33.705809 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.705790    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mk8vk"]
Sep 14 07:48:33.707595 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.707574    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 14 07:48:33.707724 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.707632    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 14 07:48:33.707791 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.707713    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 14 07:48:33.707940 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.707915    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wv9fj\""
Sep 14 07:48:33.709241 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.709221    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s6ld9"]
Sep 14 07:48:33.814820 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.814788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a90a408-7211-425a-9227-84e964f6d045-config-volume\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:33.814820 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.814822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a90a408-7211-425a-9227-84e964f6d045-tmp-dir\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:33.815048 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.814849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:33.815048 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.814930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbc0569a-8541-4a2a-b92c-80292eff74b5-cert\") pod \"ingress-canary-s6ld9\" (UID: \"bbc0569a-8541-4a2a-b92c-80292eff74b5\") " pod="openshift-ingress-canary/ingress-canary-s6ld9"
Sep 14 07:48:33.815048 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.814952    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95ch6\" (UniqueName: \"kubernetes.io/projected/9a90a408-7211-425a-9227-84e964f6d045-kube-api-access-95ch6\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:33.815048 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.814988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v76mb\" (UniqueName: \"kubernetes.io/projected/bbc0569a-8541-4a2a-b92c-80292eff74b5-kube-api-access-v76mb\") pod \"ingress-canary-s6ld9\" (UID: \"bbc0569a-8541-4a2a-b92c-80292eff74b5\") " pod="openshift-ingress-canary/ingress-canary-s6ld9"
Sep 14 07:48:33.869796 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.869757    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:33.879698 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.879673    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:48:33.879867 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.879841    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:48:33.882511 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.882487    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 07:48:33.883089 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.883071    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bd4sd\""
Sep 14 07:48:33.883244 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.883224    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 07:48:33.915733 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.915697    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbc0569a-8541-4a2a-b92c-80292eff74b5-cert\") pod \"ingress-canary-s6ld9\" (UID: \"bbc0569a-8541-4a2a-b92c-80292eff74b5\") " pod="openshift-ingress-canary/ingress-canary-s6ld9"
Sep 14 07:48:33.915860 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.915753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95ch6\" (UniqueName: \"kubernetes.io/projected/9a90a408-7211-425a-9227-84e964f6d045-kube-api-access-95ch6\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:33.915860 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.915780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v76mb\" (UniqueName: \"kubernetes.io/projected/bbc0569a-8541-4a2a-b92c-80292eff74b5-kube-api-access-v76mb\") pod \"ingress-canary-s6ld9\" (UID: \"bbc0569a-8541-4a2a-b92c-80292eff74b5\") " pod="openshift-ingress-canary/ingress-canary-s6ld9"
Sep 14 07:48:33.915860 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.915824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a90a408-7211-425a-9227-84e964f6d045-config-volume\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:33.915860 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.915849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a90a408-7211-425a-9227-84e964f6d045-tmp-dir\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:33.916055 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.915881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:33.916055 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:33.915989    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:33.916055 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:33.916048    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls podName:9a90a408-7211-425a-9227-84e964f6d045 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:34.416029915 +0000 UTC m=+35.044917525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls") pod "dns-default-mk8vk" (UID: "9a90a408-7211-425a-9227-84e964f6d045") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:33.916460 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.916430    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a90a408-7211-425a-9227-84e964f6d045-config-volume\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:33.916541 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.916437    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a90a408-7211-425a-9227-84e964f6d045-tmp-dir\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:33.920633 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.920611    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbc0569a-8541-4a2a-b92c-80292eff74b5-cert\") pod \"ingress-canary-s6ld9\" (UID: \"bbc0569a-8541-4a2a-b92c-80292eff74b5\") " pod="openshift-ingress-canary/ingress-canary-s6ld9"
Sep 14 07:48:33.928679 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.928659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95ch6\" (UniqueName: \"kubernetes.io/projected/9a90a408-7211-425a-9227-84e964f6d045-kube-api-access-95ch6\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:33.928913 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.928897    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v76mb\" (UniqueName: \"kubernetes.io/projected/bbc0569a-8541-4a2a-b92c-80292eff74b5-kube-api-access-v76mb\") pod \"ingress-canary-s6ld9\" (UID: \"bbc0569a-8541-4a2a-b92c-80292eff74b5\") " pod="openshift-ingress-canary/ingress-canary-s6ld9"
Sep 14 07:48:33.972026 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:33.971948    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:34.020242 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:34.020205    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s6ld9"
Sep 14 07:48:34.283856 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:34.283770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:34.420029 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:34.419986    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:34.420210 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:34.420167    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:34.420263 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:34.420249    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls podName:9a90a408-7211-425a-9227-84e964f6d045 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:35.420224194 +0000 UTC m=+36.049111817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls") pod "dns-default-mk8vk" (UID: "9a90a408-7211-425a-9227-84e964f6d045") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:34.880024 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:34.879987    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:48:34.882812 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:34.882786    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qxrn8\""
Sep 14 07:48:34.883507 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:34.883485    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 07:48:34.883772 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:34.883753    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 07:48:35.294693 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:35.294524    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s6ld9"]
Sep 14 07:48:35.297357 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:35.297329    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc0569a_8541_4a2a_b92c_80292eff74b5.slice/crio-a15bb98a7518c4fa49f63de8069da838bf4157e4be55ffb5f588effcee1ad0db WatchSource:0}: Error finding container a15bb98a7518c4fa49f63de8069da838bf4157e4be55ffb5f588effcee1ad0db: Status 404 returned error can't find the container with id a15bb98a7518c4fa49f63de8069da838bf4157e4be55ffb5f588effcee1ad0db
Sep 14 07:48:35.426734 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:35.426703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:35.426847 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:35.426814    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:35.426890 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:35.426873    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls podName:9a90a408-7211-425a-9227-84e964f6d045 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:37.426857824 +0000 UTC m=+38.055745439 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls") pod "dns-default-mk8vk" (UID: "9a90a408-7211-425a-9227-84e964f6d045") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:36.164833 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:36.164801    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:36.179092 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:36.179062    2563 generic.go:356] "Generic (PLEG): container finished" podID="d9f00ee5-9070-420e-9fe1-3e5e0482610e" containerID="f26f4459e7b4afc96060f575a20fe384684188037697d57a2e43985d300662cb" exitCode=0
Sep 14 07:48:36.179245 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:36.179136    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pk" event={"ID":"d9f00ee5-9070-420e-9fe1-3e5e0482610e","Type":"ContainerDied","Data":"f26f4459e7b4afc96060f575a20fe384684188037697d57a2e43985d300662cb"}
Sep 14 07:48:36.180778 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:36.180757    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s6ld9" event={"ID":"bbc0569a-8541-4a2a-b92c-80292eff74b5","Type":"ContainerStarted","Data":"a15bb98a7518c4fa49f63de8069da838bf4157e4be55ffb5f588effcee1ad0db"}
Sep 14 07:48:36.250155 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:36.250127    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:36.323974 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:36.323949    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:37.186987 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:37.186948    2563 generic.go:356] "Generic (PLEG): container finished" podID="d9f00ee5-9070-420e-9fe1-3e5e0482610e" containerID="f2a38b90e687aeb2694739f40922b455c3912556b254b2c591dc3d92fcab6442" exitCode=0
Sep 14 07:48:37.187164 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:37.186984    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pk" event={"ID":"d9f00ee5-9070-420e-9fe1-3e5e0482610e","Type":"ContainerDied","Data":"f2a38b90e687aeb2694739f40922b455c3912556b254b2c591dc3d92fcab6442"}
Sep 14 07:48:37.446110 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:37.445968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:37.446697 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:37.446152    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:37.446697 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:37.446232    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls podName:9a90a408-7211-425a-9227-84e964f6d045 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:41.446210411 +0000 UTC m=+42.075098021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls") pod "dns-default-mk8vk" (UID: "9a90a408-7211-425a-9227-84e964f6d045") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:38.192432 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:38.192391    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v22pk" event={"ID":"d9f00ee5-9070-420e-9fe1-3e5e0482610e","Type":"ContainerStarted","Data":"cede3b1d640c2f0f3bf4169f4c3a52d9b49d1638177de834ad80a721cf30bc26"}
Sep 14 07:48:38.193832 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:38.193806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s6ld9" event={"ID":"bbc0569a-8541-4a2a-b92c-80292eff74b5","Type":"ContainerStarted","Data":"0faf1c378b75c2d924ecfe9e511dfd5a41060e82eea0da47d5cfaace7240b82a"}
Sep 14 07:48:38.219248 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:38.219204    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v22pk" podStartSLOduration=9.797219823 podStartE2EDuration="38.219188816s" totalImagesPullingTime="28.421968993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:38.219026228 +0000 UTC m=+38.847913886" watchObservedRunningTime="2026-09-14 07:48:38.219188816 +0000 UTC m=+38.848076444"
Sep 14 07:48:38.236566 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:38.236523    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s6ld9" podStartSLOduration=2.793534945 podStartE2EDuration="5.236509703s" totalImagesPullingTime="2.442974758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:38.235420804 +0000 UTC m=+38.864308441" watchObservedRunningTime="2026-09-14 07:48:38.236509703 +0000 UTC m=+38.865397331"
Sep 14 07:48:38.349835 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:38.349799    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:38.413151 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:38.413128    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:38.480326 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:38.480260    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:39.250509 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:39.250478    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:39.346394 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:39.346364    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:41.470348 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:41.470304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:41.470772 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:41.470464    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:41.470772 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:41.470530    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls podName:9a90a408-7211-425a-9227-84e964f6d045 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:49.470513448 +0000 UTC m=+50.099401053 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls") pod "dns-default-mk8vk" (UID: "9a90a408-7211-425a-9227-84e964f6d045") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:49.522636 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:49.522597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:48:49.523093 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:49.522723    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:49.523093 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:49.522784    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls podName:9a90a408-7211-425a-9227-84e964f6d045 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:05.522771101 +0000 UTC m=+66.151658706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls") pod "dns-default-mk8vk" (UID: "9a90a408-7211-425a-9227-84e964f6d045") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:49.940538 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:49.940504    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"]
Sep 14 07:48:49.989206 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:49.989177    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-68c9854f4b-68nfz"]
Sep 14 07:48:49.989381 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:49.989311    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:48:49.995473 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:49.995438    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 14 07:48:49.995609 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:49.995513    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 14 07:48:49.996072 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:49.996052    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 14 07:48:49.996510 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:49.996492    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2qg6s\""
Sep 14 07:48:49.996582 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:49.996537    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 14 07:48:49.999272 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:49.999256    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"]
Sep 14 07:48:49.999345 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:49.999283    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68c9854f4b-68nfz"]
Sep 14 07:48:49.999382 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:49.999347    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.007174 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.007155    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 14 07:48:50.007264 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.007201    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 14 07:48:50.007518 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.007504    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 14 07:48:50.008001 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.007989    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 14 07:48:50.008416 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.008403    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 14 07:48:50.008974 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.008957    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 14 07:48:50.010457 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.010444    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mfd5m\""
Sep 14 07:48:50.076486 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.076461    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"]
Sep 14 07:48:50.092630 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.092608    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-jj5qw"]
Sep 14 07:48:50.092772 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.092732    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"
Sep 14 07:48:50.095511 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.095492    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 14 07:48:50.095666 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.095504    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 14 07:48:50.096146 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.096121    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-68tzb\""
Sep 14 07:48:50.096232 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.096196    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 14 07:48:50.096414 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.096399    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:48:50.103464 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.103444    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"]
Sep 14 07:48:50.103544 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.103537    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.108497 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.108477    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-jj5qw"]
Sep 14 07:48:50.114142 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.114122    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 14 07:48:50.114142 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.114136    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:48:50.114282 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.114136    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-vzzgd\""
Sep 14 07:48:50.114491 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.114476    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 14 07:48:50.114491 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.114487    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 14 07:48:50.121788 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.121770    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 14 07:48:50.125705 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.125690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-stats-auth\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.125762 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.125714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6ba25a8b-f03d-423f-b432-550618dfd5b4-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:48:50.125821 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.125803    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-default-certificate\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.125859 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.125833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prh8f\" (UniqueName: \"kubernetes.io/projected/01a4583c-ddea-4d2e-8d11-61ebef63ef24-kube-api-access-prh8f\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.125890 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.125862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:48:50.125890 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.125878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q2nf\" (UniqueName: \"kubernetes.io/projected/6ba25a8b-f03d-423f-b432-550618dfd5b4-kube-api-access-2q2nf\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:48:50.125955 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.125900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.125987 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.125957    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.153350 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.153327    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-db46994b6-qvc6c"]
Sep 14 07:48:50.165831 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.165811    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c29vb"]
Sep 14 07:48:50.165946 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.165931    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.171540 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.171439    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 14 07:48:50.172158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.172140    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 14 07:48:50.172291 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.172268    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7wdb9\""
Sep 14 07:48:50.172401 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.172219    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 14 07:48:50.178822 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.178800    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c29vb"
Sep 14 07:48:50.178822 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.178809    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c29vb"]
Sep 14 07:48:50.178998 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.178856    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db46994b6-qvc6c"]
Sep 14 07:48:50.187651 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.187619    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 14 07:48:50.189423 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.189407    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-q5bcz\""
Sep 14 07:48:50.189739 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.189723    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 14 07:48:50.189952 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.189936    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:48:50.227210 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af87c4a9-ecde-4180-bba1-da2834189fb4-serving-cert\") pod \"console-operator-696884fb9c-jj5qw\" (UID: \"af87c4a9-ecde-4180-bba1-da2834189fb4\") " pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.227210 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-default-certificate\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.227381 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prh8f\" (UniqueName: \"kubernetes.io/projected/01a4583c-ddea-4d2e-8d11-61ebef63ef24-kube-api-access-prh8f\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.227381 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76jsz\" (UniqueName: \"kubernetes.io/projected/4e45fd58-3953-4aca-89d6-d8facaaf9a28-kube-api-access-76jsz\") pod \"kube-storage-version-migrator-operator-6c8f84f897-hb2jh\" (UID: \"4e45fd58-3953-4aca-89d6-d8facaaf9a28\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"
Sep 14 07:48:50.227381 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:48:50.227381 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q2nf\" (UniqueName: \"kubernetes.io/projected/6ba25a8b-f03d-423f-b432-550618dfd5b4-kube-api-access-2q2nf\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:48:50.227381 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4e45fd58-3953-4aca-89d6-d8facaaf9a28-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-hb2jh\" (UID: \"4e45fd58-3953-4aca-89d6-d8facaaf9a28\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"
Sep 14 07:48:50.227709 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.227520    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 07:48:50.227709 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af87c4a9-ecde-4180-bba1-da2834189fb4-config\") pod \"console-operator-696884fb9c-jj5qw\" (UID: \"af87c4a9-ecde-4180-bba1-da2834189fb4\") " pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.227709 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.227599    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls podName:6ba25a8b-f03d-423f-b432-550618dfd5b4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:50.727580883 +0000 UTC m=+51.356468501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hj5zz" (UID: "6ba25a8b-f03d-423f-b432-550618dfd5b4") : secret "cluster-monitoring-operator-tls" not found
Sep 14 07:48:50.227709 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227623    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.227709 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227671    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.227709 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227692    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4e45fd58-3953-4aca-89d6-d8facaaf9a28-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-hb2jh\" (UID: \"4e45fd58-3953-4aca-89d6-d8facaaf9a28\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"
Sep 14 07:48:50.228002 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227744    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-stats-auth\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.228002 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.227754    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 07:48:50.228002 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227777    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af87c4a9-ecde-4180-bba1-da2834189fb4-trusted-ca\") pod \"console-operator-696884fb9c-jj5qw\" (UID: \"af87c4a9-ecde-4180-bba1-da2834189fb4\") " pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.228002 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.227793    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle podName:01a4583c-ddea-4d2e-8d11-61ebef63ef24 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:50.727775984 +0000 UTC m=+51.356663608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle") pod "router-default-68c9854f4b-68nfz" (UID: "01a4583c-ddea-4d2e-8d11-61ebef63ef24") : configmap references non-existent config key: service-ca.crt
Sep 14 07:48:50.228002 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.227824    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs podName:01a4583c-ddea-4d2e-8d11-61ebef63ef24 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:50.727816001 +0000 UTC m=+51.356703606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs") pod "router-default-68c9854f4b-68nfz" (UID: "01a4583c-ddea-4d2e-8d11-61ebef63ef24") : secret "router-metrics-certs-default" not found
Sep 14 07:48:50.228002 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227922    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btzg5\" (UniqueName: \"kubernetes.io/projected/af87c4a9-ecde-4180-bba1-da2834189fb4-kube-api-access-btzg5\") pod \"console-operator-696884fb9c-jj5qw\" (UID: \"af87c4a9-ecde-4180-bba1-da2834189fb4\") " pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.228002 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.227950    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6ba25a8b-f03d-423f-b432-550618dfd5b4-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:48:50.228576 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.228559    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6ba25a8b-f03d-423f-b432-550618dfd5b4-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:48:50.230769 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.230752    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-default-certificate\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.230769 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.230759    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-stats-auth\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.238600 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.238572    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prh8f\" (UniqueName: \"kubernetes.io/projected/01a4583c-ddea-4d2e-8d11-61ebef63ef24-kube-api-access-prh8f\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.239846 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.239830    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q2nf\" (UniqueName: \"kubernetes.io/projected/6ba25a8b-f03d-423f-b432-550618dfd5b4-kube-api-access-2q2nf\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:48:50.328297 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af87c4a9-ecde-4180-bba1-da2834189fb4-config\") pod \"console-operator-696884fb9c-jj5qw\" (UID: \"af87c4a9-ecde-4180-bba1-da2834189fb4\") " pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.328424 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328309    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e18e8e83-a272-478e-9fb7-3d566decfc51-trusted-ca\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.328424 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328335    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4e45fd58-3953-4aca-89d6-d8facaaf9a28-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-hb2jh\" (UID: \"4e45fd58-3953-4aca-89d6-d8facaaf9a28\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"
Sep 14 07:48:50.328424 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e18e8e83-a272-478e-9fb7-3d566decfc51-image-registry-private-configuration\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.328424 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af87c4a9-ecde-4180-bba1-da2834189fb4-trusted-ca\") pod \"console-operator-696884fb9c-jj5qw\" (UID: \"af87c4a9-ecde-4180-bba1-da2834189fb4\") " pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.328424 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328418    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btzg5\" (UniqueName: \"kubernetes.io/projected/af87c4a9-ecde-4180-bba1-da2834189fb4-kube-api-access-btzg5\") pod \"console-operator-696884fb9c-jj5qw\" (UID: \"af87c4a9-ecde-4180-bba1-da2834189fb4\") " pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.328664 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af87c4a9-ecde-4180-bba1-da2834189fb4-serving-cert\") pod \"console-operator-696884fb9c-jj5qw\" (UID: \"af87c4a9-ecde-4180-bba1-da2834189fb4\") " pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.328664 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76jsz\" (UniqueName: \"kubernetes.io/projected/4e45fd58-3953-4aca-89d6-d8facaaf9a28-kube-api-access-76jsz\") pod \"kube-storage-version-migrator-operator-6c8f84f897-hb2jh\" (UID: \"4e45fd58-3953-4aca-89d6-d8facaaf9a28\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"
Sep 14 07:48:50.328664 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4e45fd58-3953-4aca-89d6-d8facaaf9a28-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-hb2jh\" (UID: \"4e45fd58-3953-4aca-89d6-d8facaaf9a28\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"
Sep 14 07:48:50.328664 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328596    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-certificates\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.328664 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd6gz\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-kube-api-access-fd6gz\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.328664 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328662    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-bound-sa-token\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.329029 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.329029 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e18e8e83-a272-478e-9fb7-3d566decfc51-installation-pull-secrets\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.329029 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328902    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e18e8e83-a272-478e-9fb7-3d566decfc51-ca-trust-extracted\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.329029 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5q8p\" (UniqueName: \"kubernetes.io/projected/73263911-8bba-4703-ac5f-8b7bc2913a68-kube-api-access-g5q8p\") pod \"volume-data-source-validator-86695b7cc7-c29vb\" (UID: \"73263911-8bba-4703-ac5f-8b7bc2913a68\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c29vb"
Sep 14 07:48:50.329029 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.328969    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4e45fd58-3953-4aca-89d6-d8facaaf9a28-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-hb2jh\" (UID: \"4e45fd58-3953-4aca-89d6-d8facaaf9a28\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"
Sep 14 07:48:50.329217 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.329110    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af87c4a9-ecde-4180-bba1-da2834189fb4-config\") pod \"console-operator-696884fb9c-jj5qw\" (UID: \"af87c4a9-ecde-4180-bba1-da2834189fb4\") " pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.329402 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.329383    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af87c4a9-ecde-4180-bba1-da2834189fb4-trusted-ca\") pod \"console-operator-696884fb9c-jj5qw\" (UID: \"af87c4a9-ecde-4180-bba1-da2834189fb4\") " pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.330795 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.330772    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4e45fd58-3953-4aca-89d6-d8facaaf9a28-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-hb2jh\" (UID: \"4e45fd58-3953-4aca-89d6-d8facaaf9a28\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"
Sep 14 07:48:50.330884 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.330828    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af87c4a9-ecde-4180-bba1-da2834189fb4-serving-cert\") pod \"console-operator-696884fb9c-jj5qw\" (UID: \"af87c4a9-ecde-4180-bba1-da2834189fb4\") " pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.336975 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.336951    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76jsz\" (UniqueName: \"kubernetes.io/projected/4e45fd58-3953-4aca-89d6-d8facaaf9a28-kube-api-access-76jsz\") pod \"kube-storage-version-migrator-operator-6c8f84f897-hb2jh\" (UID: \"4e45fd58-3953-4aca-89d6-d8facaaf9a28\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"
Sep 14 07:48:50.337050 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.336998    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btzg5\" (UniqueName: \"kubernetes.io/projected/af87c4a9-ecde-4180-bba1-da2834189fb4-kube-api-access-btzg5\") pod \"console-operator-696884fb9c-jj5qw\" (UID: \"af87c4a9-ecde-4180-bba1-da2834189fb4\") " pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.402242 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.402220    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"
Sep 14 07:48:50.412417 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.412396    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:48:50.429533 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.429512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-certificates\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.429621 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.429540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd6gz\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-kube-api-access-fd6gz\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.429621 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.429556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-bound-sa-token\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.429621 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.429572    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.429803 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.429662    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e18e8e83-a272-478e-9fb7-3d566decfc51-installation-pull-secrets\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.429803 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.429670    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:48:50.429803 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.429739    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db46994b6-qvc6c: secret "image-registry-tls" not found
Sep 14 07:48:50.429803 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.429750    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e18e8e83-a272-478e-9fb7-3d566decfc51-ca-trust-extracted\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.429803 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.429789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5q8p\" (UniqueName: \"kubernetes.io/projected/73263911-8bba-4703-ac5f-8b7bc2913a68-kube-api-access-g5q8p\") pod \"volume-data-source-validator-86695b7cc7-c29vb\" (UID: \"73263911-8bba-4703-ac5f-8b7bc2913a68\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c29vb"
Sep 14 07:48:50.430031 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.429811    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls podName:e18e8e83-a272-478e-9fb7-3d566decfc51 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:50.929790094 +0000 UTC m=+51.558677716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls") pod "image-registry-db46994b6-qvc6c" (UID: "e18e8e83-a272-478e-9fb7-3d566decfc51") : secret "image-registry-tls" not found
Sep 14 07:48:50.430031 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.429922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e18e8e83-a272-478e-9fb7-3d566decfc51-trusted-ca\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.430031 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.429955    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e18e8e83-a272-478e-9fb7-3d566decfc51-image-registry-private-configuration\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.430160 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.430137    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e18e8e83-a272-478e-9fb7-3d566decfc51-ca-trust-extracted\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.430250 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.430231    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-certificates\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.430995 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.430973    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e18e8e83-a272-478e-9fb7-3d566decfc51-trusted-ca\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.432080 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.432063    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e18e8e83-a272-478e-9fb7-3d566decfc51-image-registry-private-configuration\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.432216 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.432183    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e18e8e83-a272-478e-9fb7-3d566decfc51-installation-pull-secrets\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.441061 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.441042    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-bound-sa-token\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.441290 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.441263    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd6gz\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-kube-api-access-fd6gz\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.441338 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.441294    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5q8p\" (UniqueName: \"kubernetes.io/projected/73263911-8bba-4703-ac5f-8b7bc2913a68-kube-api-access-g5q8p\") pod \"volume-data-source-validator-86695b7cc7-c29vb\" (UID: \"73263911-8bba-4703-ac5f-8b7bc2913a68\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c29vb"
Sep 14 07:48:50.487233 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.487165    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c29vb"
Sep 14 07:48:50.686717 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.686674    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh"]
Sep 14 07:48:50.689208 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:50.689180    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e45fd58_3953_4aca_89d6_d8facaaf9a28.slice/crio-b210e18ce7e2fa2fcb2661aee1ffbd502ff19ca7c8747bdbc24947e4c0b6ecd0 WatchSource:0}: Error finding container b210e18ce7e2fa2fcb2661aee1ffbd502ff19ca7c8747bdbc24947e4c0b6ecd0: Status 404 returned error can't find the container with id b210e18ce7e2fa2fcb2661aee1ffbd502ff19ca7c8747bdbc24947e4c0b6ecd0
Sep 14 07:48:50.715001 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.714923    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-jj5qw"]
Sep 14 07:48:50.717033 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:50.717011    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf87c4a9_ecde_4180_bba1_da2834189fb4.slice/crio-4eb2a33b7d933425e34cd59c80ffc4d049714714e50e3f55182a068fef05d8ce WatchSource:0}: Error finding container 4eb2a33b7d933425e34cd59c80ffc4d049714714e50e3f55182a068fef05d8ce: Status 404 returned error can't find the container with id 4eb2a33b7d933425e34cd59c80ffc4d049714714e50e3f55182a068fef05d8ce
Sep 14 07:48:50.731893 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.731873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.732020 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.731974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:48:50.732084 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.732045    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle podName:01a4583c-ddea-4d2e-8d11-61ebef63ef24 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:51.73202395 +0000 UTC m=+52.360911575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle") pod "router-default-68c9854f4b-68nfz" (UID: "01a4583c-ddea-4d2e-8d11-61ebef63ef24") : configmap references non-existent config key: service-ca.crt
Sep 14 07:48:50.732084 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.732074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:50.732196 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.732108    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 07:48:50.732196 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.732163    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls podName:6ba25a8b-f03d-423f-b432-550618dfd5b4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:51.732145556 +0000 UTC m=+52.361033175 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hj5zz" (UID: "6ba25a8b-f03d-423f-b432-550618dfd5b4") : secret "cluster-monitoring-operator-tls" not found
Sep 14 07:48:50.732196 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.732188    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 07:48:50.732295 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.732236    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs podName:01a4583c-ddea-4d2e-8d11-61ebef63ef24 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:51.732220394 +0000 UTC m=+52.361108000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs") pod "router-default-68c9854f4b-68nfz" (UID: "01a4583c-ddea-4d2e-8d11-61ebef63ef24") : secret "router-metrics-certs-default" not found
Sep 14 07:48:50.774561 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.774507    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c29vb"]
Sep 14 07:48:50.777249 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:48:50.777227    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73263911_8bba_4703_ac5f_8b7bc2913a68.slice/crio-47e1424bd2e1e644e163b1582d3f0af31ff5f756fd7e501c5a4dbb08f3d1c128 WatchSource:0}: Error finding container 47e1424bd2e1e644e163b1582d3f0af31ff5f756fd7e501c5a4dbb08f3d1c128: Status 404 returned error can't find the container with id 47e1424bd2e1e644e163b1582d3f0af31ff5f756fd7e501c5a4dbb08f3d1c128
Sep 14 07:48:50.933074 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:50.933043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:50.933220 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.933179    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:48:50.933220 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.933195    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db46994b6-qvc6c: secret "image-registry-tls" not found
Sep 14 07:48:50.933302 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:50.933248    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls podName:e18e8e83-a272-478e-9fb7-3d566decfc51 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:51.933233105 +0000 UTC m=+52.562120710 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls") pod "image-registry-db46994b6-qvc6c" (UID: "e18e8e83-a272-478e-9fb7-3d566decfc51") : secret "image-registry-tls" not found
Sep 14 07:48:51.224228 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:51.224193    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c29vb" event={"ID":"73263911-8bba-4703-ac5f-8b7bc2913a68","Type":"ContainerStarted","Data":"47e1424bd2e1e644e163b1582d3f0af31ff5f756fd7e501c5a4dbb08f3d1c128"}
Sep 14 07:48:51.225348 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:51.225311    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw" event={"ID":"af87c4a9-ecde-4180-bba1-da2834189fb4","Type":"ContainerStarted","Data":"4eb2a33b7d933425e34cd59c80ffc4d049714714e50e3f55182a068fef05d8ce"}
Sep 14 07:48:51.226351 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:51.226328    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh" event={"ID":"4e45fd58-3953-4aca-89d6-d8facaaf9a28","Type":"ContainerStarted","Data":"b210e18ce7e2fa2fcb2661aee1ffbd502ff19ca7c8747bdbc24947e4c0b6ecd0"}
Sep 14 07:48:51.739442 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:51.739383    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:48:51.739442 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:51.739439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:51.739993 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:51.739527    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:51.739993 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:51.739709    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle podName:01a4583c-ddea-4d2e-8d11-61ebef63ef24 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:53.739687097 +0000 UTC m=+54.368574717 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle") pod "router-default-68c9854f4b-68nfz" (UID: "01a4583c-ddea-4d2e-8d11-61ebef63ef24") : configmap references non-existent config key: service-ca.crt
Sep 14 07:48:51.740126 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:51.740092    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 07:48:51.740177 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:51.740152    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls podName:6ba25a8b-f03d-423f-b432-550618dfd5b4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:53.7401343 +0000 UTC m=+54.369021912 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hj5zz" (UID: "6ba25a8b-f03d-423f-b432-550618dfd5b4") : secret "cluster-monitoring-operator-tls" not found
Sep 14 07:48:51.740235 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:51.740210    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 07:48:51.740283 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:51.740242    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs podName:01a4583c-ddea-4d2e-8d11-61ebef63ef24 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:53.740231395 +0000 UTC m=+54.369119014 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs") pod "router-default-68c9854f4b-68nfz" (UID: "01a4583c-ddea-4d2e-8d11-61ebef63ef24") : secret "router-metrics-certs-default" not found
Sep 14 07:48:51.941412 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:51.941329    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:51.941612 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:51.941598    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:48:51.941701 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:51.941616    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db46994b6-qvc6c: secret "image-registry-tls" not found
Sep 14 07:48:51.941701 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:51.941697    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls podName:e18e8e83-a272-478e-9fb7-3d566decfc51 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:53.941682676 +0000 UTC m=+54.570570280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls") pod "image-registry-db46994b6-qvc6c" (UID: "e18e8e83-a272-478e-9fb7-3d566decfc51") : secret "image-registry-tls" not found
Sep 14 07:48:53.756459 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:53.756420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:53.756949 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:53.756514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:48:53.756949 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:53.756537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:53.756949 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:53.756596    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle podName:01a4583c-ddea-4d2e-8d11-61ebef63ef24 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:57.756577809 +0000 UTC m=+58.385465415 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle") pod "router-default-68c9854f4b-68nfz" (UID: "01a4583c-ddea-4d2e-8d11-61ebef63ef24") : configmap references non-existent config key: service-ca.crt
Sep 14 07:48:53.756949 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:53.756677    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 07:48:53.756949 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:53.756732    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls podName:6ba25a8b-f03d-423f-b432-550618dfd5b4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:57.756719194 +0000 UTC m=+58.385606814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hj5zz" (UID: "6ba25a8b-f03d-423f-b432-550618dfd5b4") : secret "cluster-monitoring-operator-tls" not found
Sep 14 07:48:53.756949 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:53.756677    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 07:48:53.756949 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:53.756767    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs podName:01a4583c-ddea-4d2e-8d11-61ebef63ef24 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:57.756758329 +0000 UTC m=+58.385645940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs") pod "router-default-68c9854f4b-68nfz" (UID: "01a4583c-ddea-4d2e-8d11-61ebef63ef24") : secret "router-metrics-certs-default" not found
Sep 14 07:48:53.958011 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:53.957975    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:53.958176 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:53.958137    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:48:53.958176 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:53.958159    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db46994b6-qvc6c: secret "image-registry-tls" not found
Sep 14 07:48:53.958249 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:53.958225    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls podName:e18e8e83-a272-478e-9fb7-3d566decfc51 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:57.958205543 +0000 UTC m=+58.587093152 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls") pod "image-registry-db46994b6-qvc6c" (UID: "e18e8e83-a272-478e-9fb7-3d566decfc51") : secret "image-registry-tls" not found
Sep 14 07:48:54.235297 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:54.235270    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/0.log"
Sep 14 07:48:54.235471 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:54.235307    2563 generic.go:356] "Generic (PLEG): container finished" podID="af87c4a9-ecde-4180-bba1-da2834189fb4" containerID="33f95a7bd20c7eb813d2c7956f46ffd45bc2bfd1da13d11f72432010e3e0b720" exitCode=255
Sep 14 07:48:54.235471 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:54.235371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw" event={"ID":"af87c4a9-ecde-4180-bba1-da2834189fb4","Type":"ContainerDied","Data":"33f95a7bd20c7eb813d2c7956f46ffd45bc2bfd1da13d11f72432010e3e0b720"}
Sep 14 07:48:54.235692 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:54.235671    2563 scope.go:117] "RemoveContainer" containerID="33f95a7bd20c7eb813d2c7956f46ffd45bc2bfd1da13d11f72432010e3e0b720"
Sep 14 07:48:54.237080 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:54.237056    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh" event={"ID":"4e45fd58-3953-4aca-89d6-d8facaaf9a28","Type":"ContainerStarted","Data":"8f06332f0efcd0bc7ce6c20770bf16a0cfb7aa967781ba1818c5a785db8f4608"}
Sep 14 07:48:54.238680 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:54.238656    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c29vb" event={"ID":"73263911-8bba-4703-ac5f-8b7bc2913a68","Type":"ContainerStarted","Data":"98cefb292a96667731d5ea2e4be363bfce3e68b7597bcfd7ef1e3ad33f9db581"}
Sep 14 07:48:54.282803 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:54.282668    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh" podStartSLOduration=1.607023061 podStartE2EDuration="4.282652524s" totalImagesPullingTime="2.675629463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:54.282480347 +0000 UTC m=+54.911367974" watchObservedRunningTime="2026-09-14 07:48:54.282652524 +0000 UTC m=+54.911540141"
Sep 14 07:48:54.301717 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:54.301672    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-c29vb" podStartSLOduration=2.689171777 podStartE2EDuration="4.301626121s" totalImagesPullingTime="1.612454344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:54.30046272 +0000 UTC m=+54.929350369" watchObservedRunningTime="2026-09-14 07:48:54.301626121 +0000 UTC m=+54.930513750"
Sep 14 07:48:55.029274 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.029210    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vmctf"]
Sep 14 07:48:55.031223 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.031204    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.034410 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.034391    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6zzx\""
Sep 14 07:48:55.034547 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.034393    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 14 07:48:55.034846 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.034830    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 14 07:48:55.035528 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.035510    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 14 07:48:55.035612 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.035582    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 14 07:48:55.044743 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.044724    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vmctf"]
Sep 14 07:48:55.167526 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.167487    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrs57\" (UniqueName: \"kubernetes.io/projected/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-api-access-hrs57\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.167700 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.167550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.167700 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.167593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/28a1ffa9-6f79-48eb-b760-bd9833f20c45-crio-socket\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.167700 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.167612    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/28a1ffa9-6f79-48eb-b760-bd9833f20c45-data-volume\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.167700 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.167637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.243037 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.243013    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 07:48:55.243411 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.243394    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/0.log"
Sep 14 07:48:55.243505 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.243431    2563 generic.go:356] "Generic (PLEG): container finished" podID="af87c4a9-ecde-4180-bba1-da2834189fb4" containerID="b02efddfabc3da4a8bd5eb2c6df8c1e22950b0be01b897dc35e99ceb1081fac1" exitCode=255
Sep 14 07:48:55.243558 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.243511    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw" event={"ID":"af87c4a9-ecde-4180-bba1-da2834189fb4","Type":"ContainerDied","Data":"b02efddfabc3da4a8bd5eb2c6df8c1e22950b0be01b897dc35e99ceb1081fac1"}
Sep 14 07:48:55.243618 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.243561    2563 scope.go:117] "RemoveContainer" containerID="33f95a7bd20c7eb813d2c7956f46ffd45bc2bfd1da13d11f72432010e3e0b720"
Sep 14 07:48:55.243864 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.243838    2563 scope.go:117] "RemoveContainer" containerID="b02efddfabc3da4a8bd5eb2c6df8c1e22950b0be01b897dc35e99ceb1081fac1"
Sep 14 07:48:55.244047 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:55.244027    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-jj5qw_openshift-console-operator(af87c4a9-ecde-4180-bba1-da2834189fb4)\"" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw" podUID="af87c4a9-ecde-4180-bba1-da2834189fb4"
Sep 14 07:48:55.267978 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.267957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrs57\" (UniqueName: \"kubernetes.io/projected/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-api-access-hrs57\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.268079 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.268005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.268079 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.268034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/28a1ffa9-6f79-48eb-b760-bd9833f20c45-crio-socket\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.268079 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.268050    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/28a1ffa9-6f79-48eb-b760-bd9833f20c45-data-volume\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.268079 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.268068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.268270 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:55.268136    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:48:55.268270 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:55.268197    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm podName:28a1ffa9-6f79-48eb-b760-bd9833f20c45 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:55.768179507 +0000 UTC m=+56.397067113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vmctf" (UID: "28a1ffa9-6f79-48eb-b760-bd9833f20c45") : configmap "kube-rbac-proxy" not found
Sep 14 07:48:55.268270 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:55.268214    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:55.268270 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.268237    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/28a1ffa9-6f79-48eb-b760-bd9833f20c45-crio-socket\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.268270 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:55.268255    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls podName:28a1ffa9-6f79-48eb-b760-bd9833f20c45 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:55.768240847 +0000 UTC m=+56.397128459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vmctf" (UID: "28a1ffa9-6f79-48eb-b760-bd9833f20c45") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:55.268491 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.268397    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/28a1ffa9-6f79-48eb-b760-bd9833f20c45-data-volume\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.281267 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.281211    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrs57\" (UniqueName: \"kubernetes.io/projected/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-api-access-hrs57\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.773226 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.773191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.773422 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:55.773343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:55.773422 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:55.773392    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:55.773422 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:55.773420    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:48:55.773564 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:55.773479    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls podName:28a1ffa9-6f79-48eb-b760-bd9833f20c45 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:56.773456654 +0000 UTC m=+57.402344276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vmctf" (UID: "28a1ffa9-6f79-48eb-b760-bd9833f20c45") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:55.773564 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:55.773501    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm podName:28a1ffa9-6f79-48eb-b760-bd9833f20c45 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:56.773491531 +0000 UTC m=+57.402379138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vmctf" (UID: "28a1ffa9-6f79-48eb-b760-bd9833f20c45") : configmap "kube-rbac-proxy" not found
Sep 14 07:48:56.247185 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:56.247161    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 07:48:56.247550 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:56.247463    2563 scope.go:117] "RemoveContainer" containerID="b02efddfabc3da4a8bd5eb2c6df8c1e22950b0be01b897dc35e99ceb1081fac1"
Sep 14 07:48:56.247621 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:56.247603    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-jj5qw_openshift-console-operator(af87c4a9-ecde-4180-bba1-da2834189fb4)\"" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw" podUID="af87c4a9-ecde-4180-bba1-da2834189fb4"
Sep 14 07:48:56.780788 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:56.780751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:56.780978 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:56.780806    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:56.780978 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:56.780879    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:48:56.780978 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:56.780942    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm podName:28a1ffa9-6f79-48eb-b760-bd9833f20c45 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:58.78092708 +0000 UTC m=+59.409814700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vmctf" (UID: "28a1ffa9-6f79-48eb-b760-bd9833f20c45") : configmap "kube-rbac-proxy" not found
Sep 14 07:48:56.781138 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:56.780992    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:56.781138 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:56.781041    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls podName:28a1ffa9-6f79-48eb-b760-bd9833f20c45 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:58.781028094 +0000 UTC m=+59.409915705 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vmctf" (UID: "28a1ffa9-6f79-48eb-b760-bd9833f20c45") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:57.790275 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:57.790238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:57.790746 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:57.790412    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle podName:01a4583c-ddea-4d2e-8d11-61ebef63ef24 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:05.790394813 +0000 UTC m=+66.419282442 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle") pod "router-default-68c9854f4b-68nfz" (UID: "01a4583c-ddea-4d2e-8d11-61ebef63ef24") : configmap references non-existent config key: service-ca.crt
Sep 14 07:48:57.790746 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:57.790457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:48:57.790746 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:57.790481    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:48:57.790746 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:57.790574    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 14 07:48:57.790746 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:57.790608    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs podName:01a4583c-ddea-4d2e-8d11-61ebef63ef24 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:05.79059925 +0000 UTC m=+66.419486854 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs") pod "router-default-68c9854f4b-68nfz" (UID: "01a4583c-ddea-4d2e-8d11-61ebef63ef24") : secret "router-metrics-certs-default" not found
Sep 14 07:48:57.790746 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:57.790611    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 07:48:57.790746 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:57.790692    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls podName:6ba25a8b-f03d-423f-b432-550618dfd5b4 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:05.790676698 +0000 UTC m=+66.419564318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hj5zz" (UID: "6ba25a8b-f03d-423f-b432-550618dfd5b4") : secret "cluster-monitoring-operator-tls" not found
Sep 14 07:48:57.980854 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:57.980825    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ts5z9_261155e3-6b52-449b-94ca-649f7210ccd9/dns-node-resolver/0.log"
Sep 14 07:48:57.991103 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:57.991080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:48:57.991255 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:57.991239    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:48:57.991319 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:57.991258    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db46994b6-qvc6c: secret "image-registry-tls" not found
Sep 14 07:48:57.991369 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:57.991317    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls podName:e18e8e83-a272-478e-9fb7-3d566decfc51 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:05.991298521 +0000 UTC m=+66.620186147 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls") pod "image-registry-db46994b6-qvc6c" (UID: "e18e8e83-a272-478e-9fb7-3d566decfc51") : secret "image-registry-tls" not found
Sep 14 07:48:58.798145 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:58.798107    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:58.798538 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:58.798163    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:48:58.798538 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:58.798232    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:48:58.798538 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:58.798298    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm podName:28a1ffa9-6f79-48eb-b760-bd9833f20c45 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:02.798283568 +0000 UTC m=+63.427171172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vmctf" (UID: "28a1ffa9-6f79-48eb-b760-bd9833f20c45") : configmap "kube-rbac-proxy" not found
Sep 14 07:48:58.798538 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:58.798300    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:58.798538 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:48:58.798337    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls podName:28a1ffa9-6f79-48eb-b760-bd9833f20c45 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:02.798326843 +0000 UTC m=+63.427214449 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vmctf" (UID: "28a1ffa9-6f79-48eb-b760-bd9833f20c45") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:58.972111 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:58.972082    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r4h2b_416afa6c-5c55-487f-8de6-200bc0be3e06/node-ca/0.log"
Sep 14 07:48:59.773576 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:48:59.772790    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s6ld9_bbc0569a-8541-4a2a-b92c-80292eff74b5/serve-healthcheck-canary/0.log"
Sep 14 07:49:00.168515 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:00.168486    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w4r2f"
Sep 14 07:49:00.374963 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:00.374934    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-hb2jh_4e45fd58-3953-4aca-89d6-d8facaaf9a28/kube-storage-version-migrator-operator/0.log"
Sep 14 07:49:00.413369 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:00.413345    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:49:00.413466 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:00.413392    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:49:00.413711 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:00.413699    2563 scope.go:117] "RemoveContainer" containerID="b02efddfabc3da4a8bd5eb2c6df8c1e22950b0be01b897dc35e99ceb1081fac1"
Sep 14 07:49:00.413858 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:00.413840    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-jj5qw_openshift-console-operator(af87c4a9-ecde-4180-bba1-da2834189fb4)\"" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw" podUID="af87c4a9-ecde-4180-bba1-da2834189fb4"
Sep 14 07:49:02.831492 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:02.831438    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:49:02.831907 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:02.831562    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:02.831907 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:02.831587    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:49:02.831907 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:02.831627    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm podName:28a1ffa9-6f79-48eb-b760-bd9833f20c45 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:10.831610845 +0000 UTC m=+71.460498455 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vmctf" (UID: "28a1ffa9-6f79-48eb-b760-bd9833f20c45") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:02.831907 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:02.831728    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:02.831907 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:02.831773    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls podName:28a1ffa9-6f79-48eb-b760-bd9833f20c45 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:10.831762859 +0000 UTC m=+71.460650465 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vmctf" (UID: "28a1ffa9-6f79-48eb-b760-bd9833f20c45") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:05.556700 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.556659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:49:05.557098 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.556752    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:49:05.558827 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.558809    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 07:49:05.559112 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.559095    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a90a408-7211-425a-9227-84e964f6d045-metrics-tls\") pod \"dns-default-mk8vk\" (UID: \"9a90a408-7211-425a-9227-84e964f6d045\") " pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:49:05.567692 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:05.567675    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 14 07:49:05.567786 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:05.567730    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs podName:c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:09.567713488 +0000 UTC m=+130.196601093 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs") pod "network-metrics-daemon-46qgp" (UID: "c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257") : secret "metrics-daemon-secret" not found
Sep 14 07:49:05.657648 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.657620    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrp8\" (UniqueName: \"kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8\") pod \"network-check-target-vpz6b\" (UID: \"e47647c7-20f1-440e-83da-65828c7f4605\") " pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:49:05.657729 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.657665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:49:05.660589 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.660571    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 07:49:05.660657 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.660613    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 07:49:05.670281 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.670253    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bee2d6cd-aca8-4008-be09-34cd9916d966-original-pull-secret\") pod \"global-pull-secret-syncer-zwmpr\" (UID: \"bee2d6cd-aca8-4008-be09-34cd9916d966\") " pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:49:05.670432 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.670418    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 07:49:05.681183 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.681162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hrp8\" (UniqueName: \"kubernetes.io/projected/e47647c7-20f1-440e-83da-65828c7f4605-kube-api-access-8hrp8\") pod \"network-check-target-vpz6b\" (UID: \"e47647c7-20f1-440e-83da-65828c7f4605\") " pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:49:05.696112 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.696094    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zwmpr"
Sep 14 07:49:05.799148 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.799117    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qxrn8\""
Sep 14 07:49:05.800999 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.800978    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:49:05.817013 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.816962    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7986b\""
Sep 14 07:49:05.825005 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.824990    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:49:05.859362 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.859330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:49:05.859516 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:05.859497    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle podName:01a4583c-ddea-4d2e-8d11-61ebef63ef24 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:21.859475816 +0000 UTC m=+82.488363425 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle") pod "router-default-68c9854f4b-68nfz" (UID: "01a4583c-ddea-4d2e-8d11-61ebef63ef24") : configmap references non-existent config key: service-ca.crt
Sep 14 07:49:05.859590 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.859528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:49:05.859590 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.859559    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:49:05.859724 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:05.859705    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:05.859778 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:05.859763    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls podName:6ba25a8b-f03d-423f-b432-550618dfd5b4 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:21.859749369 +0000 UTC m=+82.488636988 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-hj5zz" (UID: "6ba25a8b-f03d-423f-b432-550618dfd5b4") : secret "cluster-monitoring-operator-tls" not found
Sep 14 07:49:05.861811 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:05.861785    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01a4583c-ddea-4d2e-8d11-61ebef63ef24-metrics-certs\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:49:06.001611 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:06.001588    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zwmpr"]
Sep 14 07:49:06.003823 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:06.003798    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee2d6cd_aca8_4008_be09_34cd9916d966.slice/crio-90ffcbe7a18c1f35c20cc24a0c9596fdce73b24cb048334dd9ee4d5a2e6f9e2b WatchSource:0}: Error finding container 90ffcbe7a18c1f35c20cc24a0c9596fdce73b24cb048334dd9ee4d5a2e6f9e2b: Status 404 returned error can't find the container with id 90ffcbe7a18c1f35c20cc24a0c9596fdce73b24cb048334dd9ee4d5a2e6f9e2b
Sep 14 07:49:06.061570 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:06.061533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:49:06.063982 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:06.063962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls\") pod \"image-registry-db46994b6-qvc6c\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") " pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:49:06.077098 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:06.077079    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7wdb9\""
Sep 14 07:49:06.085765 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:06.085741    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:49:06.086932 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:06.086914    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vpz6b"]
Sep 14 07:49:06.088848 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:06.088825    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode47647c7_20f1_440e_83da_65828c7f4605.slice/crio-9d835337d16b51329e47bc83685a2219e764761f3ca40ae799fb50bb3c16358a WatchSource:0}: Error finding container 9d835337d16b51329e47bc83685a2219e764761f3ca40ae799fb50bb3c16358a: Status 404 returned error can't find the container with id 9d835337d16b51329e47bc83685a2219e764761f3ca40ae799fb50bb3c16358a
Sep 14 07:49:06.118433 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:06.118409    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mk8vk"]
Sep 14 07:49:06.122083 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:06.122057    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a90a408_7211_425a_9227_84e964f6d045.slice/crio-6bd627e2433a14990017fe689b7e9d086a7122e8f5c3ff63b331d59a777f7139 WatchSource:0}: Error finding container 6bd627e2433a14990017fe689b7e9d086a7122e8f5c3ff63b331d59a777f7139: Status 404 returned error can't find the container with id 6bd627e2433a14990017fe689b7e9d086a7122e8f5c3ff63b331d59a777f7139
Sep 14 07:49:06.273157 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:06.273123    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mk8vk" event={"ID":"9a90a408-7211-425a-9227-84e964f6d045","Type":"ContainerStarted","Data":"6bd627e2433a14990017fe689b7e9d086a7122e8f5c3ff63b331d59a777f7139"}
Sep 14 07:49:06.274267 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:06.274238    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zwmpr" event={"ID":"bee2d6cd-aca8-4008-be09-34cd9916d966","Type":"ContainerStarted","Data":"90ffcbe7a18c1f35c20cc24a0c9596fdce73b24cb048334dd9ee4d5a2e6f9e2b"}
Sep 14 07:49:06.275311 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:06.275291    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vpz6b" event={"ID":"e47647c7-20f1-440e-83da-65828c7f4605","Type":"ContainerStarted","Data":"9d835337d16b51329e47bc83685a2219e764761f3ca40ae799fb50bb3c16358a"}
Sep 14 07:49:06.397107 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:06.397040    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db46994b6-qvc6c"]
Sep 14 07:49:06.400032 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:06.400009    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode18e8e83_a272_478e_9fb7_3d566decfc51.slice/crio-651678dce783155ef635113b48b17076b874857462117052cfb0da05043ea930 WatchSource:0}: Error finding container 651678dce783155ef635113b48b17076b874857462117052cfb0da05043ea930: Status 404 returned error can't find the container with id 651678dce783155ef635113b48b17076b874857462117052cfb0da05043ea930
Sep 14 07:49:07.284670 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:07.284028    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db46994b6-qvc6c" event={"ID":"e18e8e83-a272-478e-9fb7-3d566decfc51","Type":"ContainerStarted","Data":"6f998d91897c2ff09df732ea86e4841ed97d5c8d4fca045e02ec98a316ba3fe3"}
Sep 14 07:49:07.284670 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:07.284083    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db46994b6-qvc6c" event={"ID":"e18e8e83-a272-478e-9fb7-3d566decfc51","Type":"ContainerStarted","Data":"651678dce783155ef635113b48b17076b874857462117052cfb0da05043ea930"}
Sep 14 07:49:07.285233 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:07.284698    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:49:07.305278 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:07.305233    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-db46994b6-qvc6c" podStartSLOduration=17.305212767 podStartE2EDuration="17.305212767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:07.304418205 +0000 UTC m=+67.933305844" watchObservedRunningTime="2026-09-14 07:49:07.305212767 +0000 UTC m=+67.934100396"
Sep 14 07:49:08.291397 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:08.291370    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mk8vk" event={"ID":"9a90a408-7211-425a-9227-84e964f6d045","Type":"ContainerStarted","Data":"3c2cc66436cab017d7ea00defaa88bd3537ca3318ca4c0bd974e952983773e4e"}
Sep 14 07:49:09.298469 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:09.298398    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mk8vk" event={"ID":"9a90a408-7211-425a-9227-84e964f6d045","Type":"ContainerStarted","Data":"5b0b6503c785641679d9ccfef11387d63dc11bd46aa5b69f14e997f300a414bc"}
Sep 14 07:49:09.298962 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:09.298549    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:49:09.318707 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:09.318660    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mk8vk" podStartSLOduration=34.344952268 podStartE2EDuration="36.318630946s" totalImagesPullingTime="1.973678678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:09.317745499 +0000 UTC m=+69.946633146" watchObservedRunningTime="2026-09-14 07:49:09.318630946 +0000 UTC m=+69.947518573"
Sep 14 07:49:10.906093 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:10.906067    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:49:10.906386 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:10.906115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:49:10.906627 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:10.906606    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/28a1ffa9-6f79-48eb-b760-bd9833f20c45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:49:10.908504 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:10.908483    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/28a1ffa9-6f79-48eb-b760-bd9833f20c45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vmctf\" (UID: \"28a1ffa9-6f79-48eb-b760-bd9833f20c45\") " pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:49:10.942871 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:10.942842    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6zzx\""
Sep 14 07:49:10.951035 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:10.951009    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vmctf"
Sep 14 07:49:11.230684 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:11.230636    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a1ffa9_6f79_48eb_b760_bd9833f20c45.slice/crio-3046d7525ed4d4fdfaae3cf9b9a1294a3e37d93a0e07c7b65844481fee37ea08 WatchSource:0}: Error finding container 3046d7525ed4d4fdfaae3cf9b9a1294a3e37d93a0e07c7b65844481fee37ea08: Status 404 returned error can't find the container with id 3046d7525ed4d4fdfaae3cf9b9a1294a3e37d93a0e07c7b65844481fee37ea08
Sep 14 07:49:11.234288 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:11.234263    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vmctf"]
Sep 14 07:49:11.307100 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:11.307073    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vpz6b" event={"ID":"e47647c7-20f1-440e-83da-65828c7f4605","Type":"ContainerStarted","Data":"486b58dfa738fbb90b4b16b6ed1a3deab5cb765e724ab3af7743d1e6727cff4f"}
Sep 14 07:49:11.307237 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:11.307144    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:49:11.308772 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:11.308748    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zwmpr" event={"ID":"bee2d6cd-aca8-4008-be09-34cd9916d966","Type":"ContainerStarted","Data":"95779ff53aa69e1461338e8209a458f4c7be4d04590afeecf26c57502c0d5f03"}
Sep 14 07:49:11.310324 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:11.310295    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vmctf" event={"ID":"28a1ffa9-6f79-48eb-b760-bd9833f20c45","Type":"ContainerStarted","Data":"91cd18eac2695216c37d4fe255641ccfe81225721fe3dea9511014db5feb65b2"}
Sep 14 07:49:11.310423 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:11.310354    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vmctf" event={"ID":"28a1ffa9-6f79-48eb-b760-bd9833f20c45","Type":"ContainerStarted","Data":"3046d7525ed4d4fdfaae3cf9b9a1294a3e37d93a0e07c7b65844481fee37ea08"}
Sep 14 07:49:11.325036 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:11.324995    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vpz6b" podStartSLOduration=66.514637138 podStartE2EDuration="1m11.324980124s" totalImagesPullingTime="4.810342986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:11.323790128 +0000 UTC m=+71.952677804" watchObservedRunningTime="2026-09-14 07:49:11.324980124 +0000 UTC m=+71.953867750"
Sep 14 07:49:11.340200 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:11.340162    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zwmpr" podStartSLOduration=66.441746447 podStartE2EDuration="1m11.340149123s" totalImagesPullingTime="4.898402676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:11.339661493 +0000 UTC m=+71.968549122" watchObservedRunningTime="2026-09-14 07:49:11.340149123 +0000 UTC m=+71.969036746"
Sep 14 07:49:12.318177 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:12.318090    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vmctf" event={"ID":"28a1ffa9-6f79-48eb-b760-bd9833f20c45","Type":"ContainerStarted","Data":"9b4fd5073ba460afaee4fe833405a5bd6502acfe8d57d0c0635039c3deae6633"}
Sep 14 07:49:13.880677 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:13.880625    2563 scope.go:117] "RemoveContainer" containerID="b02efddfabc3da4a8bd5eb2c6df8c1e22950b0be01b897dc35e99ceb1081fac1"
Sep 14 07:49:14.326961 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:14.326940    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 07:49:14.327102 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:14.327066    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw" event={"ID":"af87c4a9-ecde-4180-bba1-da2834189fb4","Type":"ContainerStarted","Data":"7c331872eb5e61310a01872c92eb0288d25d76db1e805700c8ad46ab077efc44"}
Sep 14 07:49:14.327372 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:14.327351    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:49:14.329197 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:14.329174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vmctf" event={"ID":"28a1ffa9-6f79-48eb-b760-bd9833f20c45","Type":"ContainerStarted","Data":"e30b78de377ce49a89afd6b6ef86b673cb1480791a328fafa408c62d79bdfbee"}
Sep 14 07:49:14.348009 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:14.347939    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw" podStartSLOduration=21.699435772 podStartE2EDuration="24.347921055s" totalImagesPullingTime="2.648485283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:14.347206373 +0000 UTC m=+74.976094024" watchObservedRunningTime="2026-09-14 07:49:14.347921055 +0000 UTC m=+74.976808682"
Sep 14 07:49:14.352031 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:14.352012    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-jj5qw"
Sep 14 07:49:14.372493 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:14.372438    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vmctf" podStartSLOduration=17.29903029 podStartE2EDuration="19.372424588s" totalImagesPullingTime="2.073394298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:14.37096791 +0000 UTC m=+74.999855558" watchObservedRunningTime="2026-09-14 07:49:14.372424588 +0000 UTC m=+75.001312214"
Sep 14 07:49:17.828108 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.828076    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"]
Sep 14 07:49:17.835335 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.835312    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"
Sep 14 07:49:17.838147 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.838117    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 14 07:49:17.838272 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.838120    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 14 07:49:17.838805 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.838786    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 14 07:49:17.838805 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.838803    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 14 07:49:17.850227 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.850208    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"]
Sep 14 07:49:17.872611 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.870310    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-db46994b6-qvc6c"]
Sep 14 07:49:17.946763 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.946731    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-775bff4dfb-tc9bw"]
Sep 14 07:49:17.949569 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.949553    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:17.964736 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.964712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f395ecd-8b3d-4d16-b035-3a5af0a27fd0-tmp\") pod \"klusterlet-addon-workmgr-cbfb7fb7c-7bhkj\" (UID: \"6f395ecd-8b3d-4d16-b035-3a5af0a27fd0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"
Sep 14 07:49:17.964868 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.964788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6f395ecd-8b3d-4d16-b035-3a5af0a27fd0-klusterlet-config\") pod \"klusterlet-addon-workmgr-cbfb7fb7c-7bhkj\" (UID: \"6f395ecd-8b3d-4d16-b035-3a5af0a27fd0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"
Sep 14 07:49:17.964921 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.964882    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2zwv\" (UniqueName: \"kubernetes.io/projected/6f395ecd-8b3d-4d16-b035-3a5af0a27fd0-kube-api-access-f2zwv\") pod \"klusterlet-addon-workmgr-cbfb7fb7c-7bhkj\" (UID: \"6f395ecd-8b3d-4d16-b035-3a5af0a27fd0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"
Sep 14 07:49:17.967980 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:17.967959    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-775bff4dfb-tc9bw"]
Sep 14 07:49:18.065994 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.065964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6f395ecd-8b3d-4d16-b035-3a5af0a27fd0-klusterlet-config\") pod \"klusterlet-addon-workmgr-cbfb7fb7c-7bhkj\" (UID: \"6f395ecd-8b3d-4d16-b035-3a5af0a27fd0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"
Sep 14 07:49:18.066170 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.066002    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8c625d4c-9cbb-4930-93e1-8b771503e661-installation-pull-secrets\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.066170 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.066025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c625d4c-9cbb-4930-93e1-8b771503e661-registry-tls\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.066170 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.066053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c88t2\" (UniqueName: \"kubernetes.io/projected/8c625d4c-9cbb-4930-93e1-8b771503e661-kube-api-access-c88t2\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.066170 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.066072    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2zwv\" (UniqueName: \"kubernetes.io/projected/6f395ecd-8b3d-4d16-b035-3a5af0a27fd0-kube-api-access-f2zwv\") pod \"klusterlet-addon-workmgr-cbfb7fb7c-7bhkj\" (UID: \"6f395ecd-8b3d-4d16-b035-3a5af0a27fd0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"
Sep 14 07:49:18.066170 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.066105    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8c625d4c-9cbb-4930-93e1-8b771503e661-image-registry-private-configuration\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.066170 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.066137    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c625d4c-9cbb-4930-93e1-8b771503e661-bound-sa-token\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.066170 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.066157    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f395ecd-8b3d-4d16-b035-3a5af0a27fd0-tmp\") pod \"klusterlet-addon-workmgr-cbfb7fb7c-7bhkj\" (UID: \"6f395ecd-8b3d-4d16-b035-3a5af0a27fd0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"
Sep 14 07:49:18.066488 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.066183    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8c625d4c-9cbb-4930-93e1-8b771503e661-registry-certificates\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.066488 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.066199    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8c625d4c-9cbb-4930-93e1-8b771503e661-trusted-ca\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.066488 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.066226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8c625d4c-9cbb-4930-93e1-8b771503e661-ca-trust-extracted\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.066488 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.066453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f395ecd-8b3d-4d16-b035-3a5af0a27fd0-tmp\") pod \"klusterlet-addon-workmgr-cbfb7fb7c-7bhkj\" (UID: \"6f395ecd-8b3d-4d16-b035-3a5af0a27fd0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"
Sep 14 07:49:18.068434 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.068411    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/6f395ecd-8b3d-4d16-b035-3a5af0a27fd0-klusterlet-config\") pod \"klusterlet-addon-workmgr-cbfb7fb7c-7bhkj\" (UID: \"6f395ecd-8b3d-4d16-b035-3a5af0a27fd0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"
Sep 14 07:49:18.076986 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.076960    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2zwv\" (UniqueName: \"kubernetes.io/projected/6f395ecd-8b3d-4d16-b035-3a5af0a27fd0-kube-api-access-f2zwv\") pod \"klusterlet-addon-workmgr-cbfb7fb7c-7bhkj\" (UID: \"6f395ecd-8b3d-4d16-b035-3a5af0a27fd0\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"
Sep 14 07:49:18.144159 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.144099    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"
Sep 14 07:49:18.167017 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.166990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8c625d4c-9cbb-4930-93e1-8b771503e661-installation-pull-secrets\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.167017 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.167025    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c625d4c-9cbb-4930-93e1-8b771503e661-registry-tls\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.167227 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.167205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c88t2\" (UniqueName: \"kubernetes.io/projected/8c625d4c-9cbb-4930-93e1-8b771503e661-kube-api-access-c88t2\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.167284 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.167250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8c625d4c-9cbb-4930-93e1-8b771503e661-image-registry-private-configuration\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.167284 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.167278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c625d4c-9cbb-4930-93e1-8b771503e661-bound-sa-token\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.167386 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.167310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8c625d4c-9cbb-4930-93e1-8b771503e661-registry-certificates\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.167386 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.167335    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8c625d4c-9cbb-4930-93e1-8b771503e661-trusted-ca\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.167509 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.167385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8c625d4c-9cbb-4930-93e1-8b771503e661-ca-trust-extracted\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.167930 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.167906    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8c625d4c-9cbb-4930-93e1-8b771503e661-ca-trust-extracted\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.168575 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.168540    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8c625d4c-9cbb-4930-93e1-8b771503e661-trusted-ca\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.168575 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.168551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8c625d4c-9cbb-4930-93e1-8b771503e661-registry-certificates\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.169597 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.169571    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8c625d4c-9cbb-4930-93e1-8b771503e661-registry-tls\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.169733 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.169713    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8c625d4c-9cbb-4930-93e1-8b771503e661-installation-pull-secrets\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.169987 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.169957    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8c625d4c-9cbb-4930-93e1-8b771503e661-image-registry-private-configuration\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.177686 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.177668    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8c625d4c-9cbb-4930-93e1-8b771503e661-bound-sa-token\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.178166 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.178145    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c88t2\" (UniqueName: \"kubernetes.io/projected/8c625d4c-9cbb-4930-93e1-8b771503e661-kube-api-access-c88t2\") pod \"image-registry-775bff4dfb-tc9bw\" (UID: \"8c625d4c-9cbb-4930-93e1-8b771503e661\") " pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.258147 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.258114    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:18.415837 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.415814    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"]
Sep 14 07:49:18.418147 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:18.418120    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f395ecd_8b3d_4d16_b035_3a5af0a27fd0.slice/crio-91feb3cc67b4b3f1abc58c06aa058bcfbb0b1f641e5cb0c715fc3b1d206d3d50 WatchSource:0}: Error finding container 91feb3cc67b4b3f1abc58c06aa058bcfbb0b1f641e5cb0c715fc3b1d206d3d50: Status 404 returned error can't find the container with id 91feb3cc67b4b3f1abc58c06aa058bcfbb0b1f641e5cb0c715fc3b1d206d3d50
Sep 14 07:49:18.526475 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:18.526442    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-775bff4dfb-tc9bw"]
Sep 14 07:49:18.529124 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:18.529100    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c625d4c_9cbb_4930_93e1_8b771503e661.slice/crio-f112030d3a642da1301000c017a872984c88b822cced33071a03ab39cf617414 WatchSource:0}: Error finding container f112030d3a642da1301000c017a872984c88b822cced33071a03ab39cf617414: Status 404 returned error can't find the container with id f112030d3a642da1301000c017a872984c88b822cced33071a03ab39cf617414
Sep 14 07:49:19.304477 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:19.304442    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mk8vk"
Sep 14 07:49:19.348074 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:19.348041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj" event={"ID":"6f395ecd-8b3d-4d16-b035-3a5af0a27fd0","Type":"ContainerStarted","Data":"91feb3cc67b4b3f1abc58c06aa058bcfbb0b1f641e5cb0c715fc3b1d206d3d50"}
Sep 14 07:49:19.351118 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:19.350287    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw" event={"ID":"8c625d4c-9cbb-4930-93e1-8b771503e661","Type":"ContainerStarted","Data":"8156e48b66c7a4f2dabf1343c810ecf1307d21428c3e3d9f939ede68556361a2"}
Sep 14 07:49:19.351118 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:19.350337    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw" event={"ID":"8c625d4c-9cbb-4930-93e1-8b771503e661","Type":"ContainerStarted","Data":"f112030d3a642da1301000c017a872984c88b822cced33071a03ab39cf617414"}
Sep 14 07:49:19.351118 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:19.351064    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:19.373006 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:19.372943    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw" podStartSLOduration=2.372923151 podStartE2EDuration="2.372923151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:19.371547143 +0000 UTC m=+80.000434792" watchObservedRunningTime="2026-09-14 07:49:19.372923151 +0000 UTC m=+80.001810774"
Sep 14 07:49:21.901880 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:21.901622    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:49:21.901880 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:21.901736    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:49:21.902418 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:21.902394    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01a4583c-ddea-4d2e-8d11-61ebef63ef24-service-ca-bundle\") pod \"router-default-68c9854f4b-68nfz\" (UID: \"01a4583c-ddea-4d2e-8d11-61ebef63ef24\") " pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:49:21.904434 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:21.904408    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6ba25a8b-f03d-423f-b432-550618dfd5b4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-hj5zz\" (UID: \"6ba25a8b-f03d-423f-b432-550618dfd5b4\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:49:22.102778 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:22.102747    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2qg6s\""
Sep 14 07:49:22.108986 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:22.108966    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mfd5m\""
Sep 14 07:49:22.111067 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:22.111053    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"
Sep 14 07:49:22.117130 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:22.117110    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:49:22.365058 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:22.365021    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj" event={"ID":"6f395ecd-8b3d-4d16-b035-3a5af0a27fd0","Type":"ContainerStarted","Data":"bbf03f3dcca1f387c558bc3fb32fd979e73b472fb56e7950c1e9df928691670b"}
Sep 14 07:49:22.365217 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:22.365197    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"
Sep 14 07:49:22.366870 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:22.366848    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj"
Sep 14 07:49:22.388138 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:22.388096    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-cbfb7fb7c-7bhkj" podStartSLOduration=1.825517724 podStartE2EDuration="5.388081997s" totalImagesPullingTime="3.562564273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:22.387095991 +0000 UTC m=+83.015983642" watchObservedRunningTime="2026-09-14 07:49:22.388081997 +0000 UTC m=+83.016969849"
Sep 14 07:49:22.400010 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:22.399989    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz"]
Sep 14 07:49:22.402128 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:22.402111    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba25a8b_f03d_423f_b432_550618dfd5b4.slice/crio-69252e120cc15f11910822db30c95bfb3d651fcea9631dbf47623725c8e15fd6 WatchSource:0}: Error finding container 69252e120cc15f11910822db30c95bfb3d651fcea9631dbf47623725c8e15fd6: Status 404 returned error can't find the container with id 69252e120cc15f11910822db30c95bfb3d651fcea9631dbf47623725c8e15fd6
Sep 14 07:49:22.405283 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:22.405264    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68c9854f4b-68nfz"]
Sep 14 07:49:22.408206 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:22.408182    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a4583c_ddea_4d2e_8d11_61ebef63ef24.slice/crio-7a6a98240799967908e0793b363a59148baa236c3a0c2acad6fef14b33d44fb8 WatchSource:0}: Error finding container 7a6a98240799967908e0793b363a59148baa236c3a0c2acad6fef14b33d44fb8: Status 404 returned error can't find the container with id 7a6a98240799967908e0793b363a59148baa236c3a0c2acad6fef14b33d44fb8
Sep 14 07:49:23.372092 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:23.372053    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68c9854f4b-68nfz" event={"ID":"01a4583c-ddea-4d2e-8d11-61ebef63ef24","Type":"ContainerStarted","Data":"17850bff1fae7107c1b6bff7ff5ff13b980e4e341f04a1616a4932c46d366892"}
Sep 14 07:49:23.372692 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:23.372667    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68c9854f4b-68nfz" event={"ID":"01a4583c-ddea-4d2e-8d11-61ebef63ef24","Type":"ContainerStarted","Data":"7a6a98240799967908e0793b363a59148baa236c3a0c2acad6fef14b33d44fb8"}
Sep 14 07:49:23.375396 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:23.375346    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz" event={"ID":"6ba25a8b-f03d-423f-b432-550618dfd5b4","Type":"ContainerStarted","Data":"69252e120cc15f11910822db30c95bfb3d651fcea9631dbf47623725c8e15fd6"}
Sep 14 07:49:23.395307 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:23.395255    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-68c9854f4b-68nfz" podStartSLOduration=34.395239034 podStartE2EDuration="34.395239034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:23.393729036 +0000 UTC m=+84.022616684" watchObservedRunningTime="2026-09-14 07:49:23.395239034 +0000 UTC m=+84.024126665"
Sep 14 07:49:24.117752 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:24.117715    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:49:24.120695 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:24.120670    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:49:24.379258 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:24.379181    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:49:24.380727 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:24.380695    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-68c9854f4b-68nfz"
Sep 14 07:49:25.384321 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:25.384285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz" event={"ID":"6ba25a8b-f03d-423f-b432-550618dfd5b4","Type":"ContainerStarted","Data":"c6784eb6365fca40580a60504c7b75b410dcf78658f0e64a912ccfe35b38aafe"}
Sep 14 07:49:25.403794 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:25.403748    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-hj5zz" podStartSLOduration=34.275777109 podStartE2EDuration="36.403731578s" totalImagesPullingTime="2.127954469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:25.402323089 +0000 UTC m=+86.031210737" watchObservedRunningTime="2026-09-14 07:49:25.403731578 +0000 UTC m=+86.032619205"
Sep 14 07:49:27.875757 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:27.875728    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:49:41.364953 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:41.364921    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-775bff4dfb-tc9bw"
Sep 14 07:49:42.321145 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:42.321114    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vpz6b"
Sep 14 07:49:42.891772 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:42.891728    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-db46994b6-qvc6c" podUID="e18e8e83-a272-478e-9fb7-3d566decfc51" containerName="registry" containerID="cri-o://6f998d91897c2ff09df732ea86e4841ed97d5c8d4fca045e02ec98a316ba3fe3" gracePeriod=30
Sep 14 07:49:43.165406 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.165331    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:49:43.263271 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.263240    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls\") pod \"e18e8e83-a272-478e-9fb7-3d566decfc51\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") "
Sep 14 07:49:43.263489 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.263284    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-bound-sa-token\") pod \"e18e8e83-a272-478e-9fb7-3d566decfc51\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") "
Sep 14 07:49:43.263489 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.263309    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-certificates\") pod \"e18e8e83-a272-478e-9fb7-3d566decfc51\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") "
Sep 14 07:49:43.263489 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.263328    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e18e8e83-a272-478e-9fb7-3d566decfc51-installation-pull-secrets\") pod \"e18e8e83-a272-478e-9fb7-3d566decfc51\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") "
Sep 14 07:49:43.263489 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.263357    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fd6gz\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-kube-api-access-fd6gz\") pod \"e18e8e83-a272-478e-9fb7-3d566decfc51\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") "
Sep 14 07:49:43.263489 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.263395    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e18e8e83-a272-478e-9fb7-3d566decfc51-trusted-ca\") pod \"e18e8e83-a272-478e-9fb7-3d566decfc51\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") "
Sep 14 07:49:43.263489 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.263421    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e18e8e83-a272-478e-9fb7-3d566decfc51-ca-trust-extracted\") pod \"e18e8e83-a272-478e-9fb7-3d566decfc51\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") "
Sep 14 07:49:43.263489 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.263453    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e18e8e83-a272-478e-9fb7-3d566decfc51-image-registry-private-configuration\") pod \"e18e8e83-a272-478e-9fb7-3d566decfc51\" (UID: \"e18e8e83-a272-478e-9fb7-3d566decfc51\") "
Sep 14 07:49:43.264135 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.263814    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e18e8e83-a272-478e-9fb7-3d566decfc51" (UID: "e18e8e83-a272-478e-9fb7-3d566decfc51"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:49:43.264728 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.264684    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e18e8e83-a272-478e-9fb7-3d566decfc51-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e18e8e83-a272-478e-9fb7-3d566decfc51" (UID: "e18e8e83-a272-478e-9fb7-3d566decfc51"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:49:43.266008 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.265971    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e18e8e83-a272-478e-9fb7-3d566decfc51" (UID: "e18e8e83-a272-478e-9fb7-3d566decfc51"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:49:43.266142 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.266119    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e18e8e83-a272-478e-9fb7-3d566decfc51-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e18e8e83-a272-478e-9fb7-3d566decfc51" (UID: "e18e8e83-a272-478e-9fb7-3d566decfc51"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:49:43.266234 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.266191    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-kube-api-access-fd6gz" (OuterVolumeSpecName: "kube-api-access-fd6gz") pod "e18e8e83-a272-478e-9fb7-3d566decfc51" (UID: "e18e8e83-a272-478e-9fb7-3d566decfc51"). InnerVolumeSpecName "kube-api-access-fd6gz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:49:43.266300 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.266275    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e18e8e83-a272-478e-9fb7-3d566decfc51" (UID: "e18e8e83-a272-478e-9fb7-3d566decfc51"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:49:43.266473 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.266454    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e18e8e83-a272-478e-9fb7-3d566decfc51-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e18e8e83-a272-478e-9fb7-3d566decfc51" (UID: "e18e8e83-a272-478e-9fb7-3d566decfc51"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:49:43.274851 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.274824    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e18e8e83-a272-478e-9fb7-3d566decfc51-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e18e8e83-a272-478e-9fb7-3d566decfc51" (UID: "e18e8e83-a272-478e-9fb7-3d566decfc51"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:49:43.364236 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.364208    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e18e8e83-a272-478e-9fb7-3d566decfc51-trusted-ca\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:49:43.364236 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.364232    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e18e8e83-a272-478e-9fb7-3d566decfc51-ca-trust-extracted\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:49:43.364236 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.364242    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e18e8e83-a272-478e-9fb7-3d566decfc51-image-registry-private-configuration\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:49:43.364426 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.364251    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-tls\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:49:43.364426 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.364259    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-bound-sa-token\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:49:43.364426 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.364267    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e18e8e83-a272-478e-9fb7-3d566decfc51-registry-certificates\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:49:43.364426 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.364275    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e18e8e83-a272-478e-9fb7-3d566decfc51-installation-pull-secrets\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:49:43.364426 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.364284    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd6gz\" (UniqueName: \"kubernetes.io/projected/e18e8e83-a272-478e-9fb7-3d566decfc51-kube-api-access-fd6gz\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:49:43.449733 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.449655    2563 generic.go:356] "Generic (PLEG): container finished" podID="e18e8e83-a272-478e-9fb7-3d566decfc51" containerID="6f998d91897c2ff09df732ea86e4841ed97d5c8d4fca045e02ec98a316ba3fe3" exitCode=0
Sep 14 07:49:43.449733 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.449704    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db46994b6-qvc6c"
Sep 14 07:49:43.449887 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.449734    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db46994b6-qvc6c" event={"ID":"e18e8e83-a272-478e-9fb7-3d566decfc51","Type":"ContainerDied","Data":"6f998d91897c2ff09df732ea86e4841ed97d5c8d4fca045e02ec98a316ba3fe3"}
Sep 14 07:49:43.449887 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.449801    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db46994b6-qvc6c" event={"ID":"e18e8e83-a272-478e-9fb7-3d566decfc51","Type":"ContainerDied","Data":"651678dce783155ef635113b48b17076b874857462117052cfb0da05043ea930"}
Sep 14 07:49:43.449887 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.449821    2563 scope.go:117] "RemoveContainer" containerID="6f998d91897c2ff09df732ea86e4841ed97d5c8d4fca045e02ec98a316ba3fe3"
Sep 14 07:49:43.458291 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.458277    2563 scope.go:117] "RemoveContainer" containerID="6f998d91897c2ff09df732ea86e4841ed97d5c8d4fca045e02ec98a316ba3fe3"
Sep 14 07:49:43.458573 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:43.458553    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f998d91897c2ff09df732ea86e4841ed97d5c8d4fca045e02ec98a316ba3fe3\": container with ID starting with 6f998d91897c2ff09df732ea86e4841ed97d5c8d4fca045e02ec98a316ba3fe3 not found: ID does not exist" containerID="6f998d91897c2ff09df732ea86e4841ed97d5c8d4fca045e02ec98a316ba3fe3"
Sep 14 07:49:43.458625 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.458592    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f998d91897c2ff09df732ea86e4841ed97d5c8d4fca045e02ec98a316ba3fe3"} err="failed to get container status \"6f998d91897c2ff09df732ea86e4841ed97d5c8d4fca045e02ec98a316ba3fe3\": rpc error: code = NotFound desc = could not find container \"6f998d91897c2ff09df732ea86e4841ed97d5c8d4fca045e02ec98a316ba3fe3\": container with ID starting with 6f998d91897c2ff09df732ea86e4841ed97d5c8d4fca045e02ec98a316ba3fe3 not found: ID does not exist"
Sep 14 07:49:43.471036 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.471016    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-db46994b6-qvc6c"]
Sep 14 07:49:43.477518 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.477500    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-db46994b6-qvc6c"]
Sep 14 07:49:43.883613 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:43.883583    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e18e8e83-a272-478e-9fb7-3d566decfc51" path="/var/lib/kubelet/pods/e18e8e83-a272-478e-9fb7-3d566decfc51/volumes"
Sep 14 07:49:44.436887 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.436855    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"]
Sep 14 07:49:44.437320 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.437091    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e18e8e83-a272-478e-9fb7-3d566decfc51" containerName="registry"
Sep 14 07:49:44.437320 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.437102    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e18e8e83-a272-478e-9fb7-3d566decfc51" containerName="registry"
Sep 14 07:49:44.437320 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.437164    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e18e8e83-a272-478e-9fb7-3d566decfc51" containerName="registry"
Sep 14 07:49:44.471608 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.471571    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"]
Sep 14 07:49:44.471797 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.471711    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:44.473801 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.473776    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 14 07:49:44.473929 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.473787    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 14 07:49:44.474414 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.474397    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 14 07:49:44.474494 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.474429    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-lg6q7\""
Sep 14 07:49:44.495046 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.495021    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"]
Sep 14 07:49:44.495150 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.495064    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"]
Sep 14 07:49:44.495150 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.495097    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8hbxw"]
Sep 14 07:49:44.495150 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.495139    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.497327 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.497310    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 14 07:49:44.497416 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.497330    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 14 07:49:44.497416 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.497333    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 14 07:49:44.497497 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.497475    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-n5rjb\""
Sep 14 07:49:44.517233 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.517211    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.519537 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.519516    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 14 07:49:44.519653 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.519516    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6jx2x\""
Sep 14 07:49:44.519653 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.519583    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 14 07:49:44.520060 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.520040    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 14 07:49:44.572692 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.572665    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39196832-8ece-4519-b222-0f5d5043422a-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-9wdh8\" (UID: \"39196832-8ece-4519-b222-0f5d5043422a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:44.572796 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.572714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lxvt\" (UniqueName: \"kubernetes.io/projected/39196832-8ece-4519-b222-0f5d5043422a-kube-api-access-5lxvt\") pod \"openshift-state-metrics-67dcf8c979-9wdh8\" (UID: \"39196832-8ece-4519-b222-0f5d5043422a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:44.572796 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.572758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39196832-8ece-4519-b222-0f5d5043422a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-9wdh8\" (UID: \"39196832-8ece-4519-b222-0f5d5043422a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:44.572870 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.572818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39196832-8ece-4519-b222-0f5d5043422a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-9wdh8\" (UID: \"39196832-8ece-4519-b222-0f5d5043422a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:44.673617 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.673591    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04790d5e-1c25-4b84-8e27-b4655f6e50ee-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.673764 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.673626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39196832-8ece-4519-b222-0f5d5043422a-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-9wdh8\" (UID: \"39196832-8ece-4519-b222-0f5d5043422a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:44.673764 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.673660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-accelerators-collector-config\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.673764 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.673677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqvjv\" (UniqueName: \"kubernetes.io/projected/4034874d-df6a-4e24-a248-fe735722dc7b-kube-api-access-bqvjv\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.673764 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.673740    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.673979 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.673775    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxfgg\" (UniqueName: \"kubernetes.io/projected/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-api-access-xxfgg\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.673979 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.673806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-textfile\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.673979 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.673834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4034874d-df6a-4e24-a248-fe735722dc7b-metrics-client-ca\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.673979 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.673869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lxvt\" (UniqueName: \"kubernetes.io/projected/39196832-8ece-4519-b222-0f5d5043422a-kube-api-access-5lxvt\") pod \"openshift-state-metrics-67dcf8c979-9wdh8\" (UID: \"39196832-8ece-4519-b222-0f5d5043422a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:44.673979 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.673904    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4034874d-df6a-4e24-a248-fe735722dc7b-root\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.673979 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.673928    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4034874d-df6a-4e24-a248-fe735722dc7b-sys\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.673979 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.673956    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.674330 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.673988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-wtmp\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.674330 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.674017    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.674330 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.674052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/04790d5e-1c25-4b84-8e27-b4655f6e50ee-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.674330 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.674086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-tls\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.674330 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.674127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39196832-8ece-4519-b222-0f5d5043422a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-9wdh8\" (UID: \"39196832-8ece-4519-b222-0f5d5043422a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:44.674330 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.674174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.674330 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.674204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39196832-8ece-4519-b222-0f5d5043422a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-9wdh8\" (UID: \"39196832-8ece-4519-b222-0f5d5043422a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:44.674330 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.674272    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39196832-8ece-4519-b222-0f5d5043422a-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-9wdh8\" (UID: \"39196832-8ece-4519-b222-0f5d5043422a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:44.674330 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:44.674278    2563 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 14 07:49:44.674751 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:44.674396    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39196832-8ece-4519-b222-0f5d5043422a-openshift-state-metrics-tls podName:39196832-8ece-4519-b222-0f5d5043422a nodeName:}" failed. No retries permitted until 2026-09-14 07:49:45.174379509 +0000 UTC m=+105.803267125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/39196832-8ece-4519-b222-0f5d5043422a-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-9wdh8" (UID: "39196832-8ece-4519-b222-0f5d5043422a") : secret "openshift-state-metrics-tls" not found
Sep 14 07:49:44.676664 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.676625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39196832-8ece-4519-b222-0f5d5043422a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-9wdh8\" (UID: \"39196832-8ece-4519-b222-0f5d5043422a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:44.685780 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.685754    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lxvt\" (UniqueName: \"kubernetes.io/projected/39196832-8ece-4519-b222-0f5d5043422a-kube-api-access-5lxvt\") pod \"openshift-state-metrics-67dcf8c979-9wdh8\" (UID: \"39196832-8ece-4519-b222-0f5d5043422a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:44.775073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04790d5e-1c25-4b84-8e27-b4655f6e50ee-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.775073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775045    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-accelerators-collector-config\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775073 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqvjv\" (UniqueName: \"kubernetes.io/projected/4034874d-df6a-4e24-a248-fe735722dc7b-kube-api-access-bqvjv\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775324 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.775324 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775179    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxfgg\" (UniqueName: \"kubernetes.io/projected/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-api-access-xxfgg\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.775324 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-textfile\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775324 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4034874d-df6a-4e24-a248-fe735722dc7b-metrics-client-ca\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775324 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775277    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4034874d-df6a-4e24-a248-fe735722dc7b-root\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775324 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4034874d-df6a-4e24-a248-fe735722dc7b-sys\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775370    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-wtmp\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775383    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4034874d-df6a-4e24-a248-fe735722dc7b-root\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775398    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:44.775471    2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775503    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4034874d-df6a-4e24-a248-fe735722dc7b-sys\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:44.775528    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-state-metrics-tls podName:04790d5e-1c25-4b84-8e27-b4655f6e50ee nodeName:}" failed. No retries permitted until 2026-09-14 07:49:45.275507456 +0000 UTC m=+105.904395065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-2gs86" (UID: "04790d5e-1c25-4b84-8e27-b4655f6e50ee") : secret "kube-state-metrics-tls" not found
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/04790d5e-1c25-4b84-8e27-b4655f6e50ee-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775550    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-textfile\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775567    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-wtmp\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775584    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-tls\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:44.775684    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775683    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-accelerators-collector-config\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.775758 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:49:44.775750    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-tls podName:4034874d-df6a-4e24-a248-fe735722dc7b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:45.275732447 +0000 UTC m=+105.904620058 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-tls") pod "node-exporter-8hbxw" (UID: "4034874d-df6a-4e24-a248-fe735722dc7b") : secret "node-exporter-tls" not found
Sep 14 07:49:44.776485 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.775786    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4034874d-df6a-4e24-a248-fe735722dc7b-metrics-client-ca\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.776485 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.776031    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/04790d5e-1c25-4b84-8e27-b4655f6e50ee-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.776485 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.776201    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.776485 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.776332    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04790d5e-1c25-4b84-8e27-b4655f6e50ee-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.777870 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.777848    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.777870 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.777858    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:44.783569 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.783550    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqvjv\" (UniqueName: \"kubernetes.io/projected/4034874d-df6a-4e24-a248-fe735722dc7b-kube-api-access-bqvjv\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:44.783745 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:44.783725    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxfgg\" (UniqueName: \"kubernetes.io/projected/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-api-access-xxfgg\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:45.179184 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:45.179148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39196832-8ece-4519-b222-0f5d5043422a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-9wdh8\" (UID: \"39196832-8ece-4519-b222-0f5d5043422a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:45.181605 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:45.181582    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39196832-8ece-4519-b222-0f5d5043422a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-9wdh8\" (UID: \"39196832-8ece-4519-b222-0f5d5043422a\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:45.279591 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:45.279554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:45.279780 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:45.279609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-tls\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:45.281909 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:45.281887    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4034874d-df6a-4e24-a248-fe735722dc7b-node-exporter-tls\") pod \"node-exporter-8hbxw\" (UID: \"4034874d-df6a-4e24-a248-fe735722dc7b\") " pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:45.282008 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:45.281928    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/04790d5e-1c25-4b84-8e27-b4655f6e50ee-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-2gs86\" (UID: \"04790d5e-1c25-4b84-8e27-b4655f6e50ee\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:45.380717 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:45.380691    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"
Sep 14 07:49:45.403876 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:45.403848    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"
Sep 14 07:49:45.426267 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:45.426240    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8hbxw"
Sep 14 07:49:45.442052 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:45.442008    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4034874d_df6a_4e24_a248_fe735722dc7b.slice/crio-ab784f8abfa39d5a0f13452f257df98d8c08d2e2952e9ee4ca280efe4995c268 WatchSource:0}: Error finding container ab784f8abfa39d5a0f13452f257df98d8c08d2e2952e9ee4ca280efe4995c268: Status 404 returned error can't find the container with id ab784f8abfa39d5a0f13452f257df98d8c08d2e2952e9ee4ca280efe4995c268
Sep 14 07:49:45.461526 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:45.461491    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8hbxw" event={"ID":"4034874d-df6a-4e24-a248-fe735722dc7b","Type":"ContainerStarted","Data":"ab784f8abfa39d5a0f13452f257df98d8c08d2e2952e9ee4ca280efe4995c268"}
Sep 14 07:49:45.664103 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:45.664076    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8"]
Sep 14 07:49:45.665937 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:45.665911    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39196832_8ece_4519_b222_0f5d5043422a.slice/crio-726df386819a154d109d8396c60706371b89c543b18164ce4829073ae8732c23 WatchSource:0}: Error finding container 726df386819a154d109d8396c60706371b89c543b18164ce4829073ae8732c23: Status 404 returned error can't find the container with id 726df386819a154d109d8396c60706371b89c543b18164ce4829073ae8732c23
Sep 14 07:49:45.690905 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:45.690848    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86"]
Sep 14 07:49:45.694150 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:45.694089    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04790d5e_1c25_4b84_8e27_b4655f6e50ee.slice/crio-e001deb16cc96b404f221ee3db59532ae7c089a0cf5250b316c73b3c9521f375 WatchSource:0}: Error finding container e001deb16cc96b404f221ee3db59532ae7c089a0cf5250b316c73b3c9521f375: Status 404 returned error can't find the container with id e001deb16cc96b404f221ee3db59532ae7c089a0cf5250b316c73b3c9521f375
Sep 14 07:49:46.468669 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:46.468601    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8" event={"ID":"39196832-8ece-4519-b222-0f5d5043422a","Type":"ContainerStarted","Data":"70ab6e7df6181abc47587a5017a697fcea74c7431a5028137f54923daef5f7bb"}
Sep 14 07:49:46.469134 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:46.468694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8" event={"ID":"39196832-8ece-4519-b222-0f5d5043422a","Type":"ContainerStarted","Data":"c50d08c2db48c9282c4007ab6e635f746bf0396079160d560d95db1ee06afb65"}
Sep 14 07:49:46.469134 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:46.468725    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8" event={"ID":"39196832-8ece-4519-b222-0f5d5043422a","Type":"ContainerStarted","Data":"726df386819a154d109d8396c60706371b89c543b18164ce4829073ae8732c23"}
Sep 14 07:49:46.470307 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:46.470277    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86" event={"ID":"04790d5e-1c25-4b84-8e27-b4655f6e50ee","Type":"ContainerStarted","Data":"e001deb16cc96b404f221ee3db59532ae7c089a0cf5250b316c73b3c9521f375"}
Sep 14 07:49:47.478357 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:47.478314    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86" event={"ID":"04790d5e-1c25-4b84-8e27-b4655f6e50ee","Type":"ContainerStarted","Data":"100b876899e827af317e14f6f43b693ebe7c9daabd9bbcda399dd8a3b2c12ca5"}
Sep 14 07:49:47.478876 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:47.478395    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86" event={"ID":"04790d5e-1c25-4b84-8e27-b4655f6e50ee","Type":"ContainerStarted","Data":"4595ef4d3480b91d15f596ba866d4ee3eeb458e9c71d80c77334db444ef60727"}
Sep 14 07:49:47.478876 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:47.478424    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86" event={"ID":"04790d5e-1c25-4b84-8e27-b4655f6e50ee","Type":"ContainerStarted","Data":"4535b14092aa6dbf70822bc1692d4773d4b95710ea5e8e26ffc5a9943cbcb5bf"}
Sep 14 07:49:47.480449 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:47.480417    2563 generic.go:356] "Generic (PLEG): container finished" podID="4034874d-df6a-4e24-a248-fe735722dc7b" containerID="b120a11dec795949267adb4a359cad69891665bcae53bf3c3d14c94a6edb0bd8" exitCode=0
Sep 14 07:49:47.480598 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:47.480521    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8hbxw" event={"ID":"4034874d-df6a-4e24-a248-fe735722dc7b","Type":"ContainerDied","Data":"b120a11dec795949267adb4a359cad69891665bcae53bf3c3d14c94a6edb0bd8"}
Sep 14 07:49:47.484820 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:47.484694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8" event={"ID":"39196832-8ece-4519-b222-0f5d5043422a","Type":"ContainerStarted","Data":"883db348154a81ff06ee6e94726d54633adb869e833455194d2be28e1505faa3"}
Sep 14 07:49:47.526094 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:47.526037    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-2gs86" podStartSLOduration=2.052300198 podStartE2EDuration="3.52601586s" totalImagesPullingTime="1.473715662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:47.500487865 +0000 UTC m=+108.129375516" watchObservedRunningTime="2026-09-14 07:49:47.52601586 +0000 UTC m=+108.154903489"
Sep 14 07:49:47.549004 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:47.548944    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-9wdh8" podStartSLOduration=2.180699967 podStartE2EDuration="3.548927071s" totalImagesPullingTime="1.368227104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:47.547944341 +0000 UTC m=+108.176831989" watchObservedRunningTime="2026-09-14 07:49:47.548927071 +0000 UTC m=+108.177814699"
Sep 14 07:49:48.491518 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.491474    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8hbxw" event={"ID":"4034874d-df6a-4e24-a248-fe735722dc7b","Type":"ContainerStarted","Data":"203fec27548335f7ef0cf07048978eec3d5cd2a6ba14198fceee134821e9ba5d"}
Sep 14 07:49:48.491916 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.491580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8hbxw" event={"ID":"4034874d-df6a-4e24-a248-fe735722dc7b","Type":"ContainerStarted","Data":"e01ab1a047f1d2f0176d5d0ba54a58146c2365d4a3cde3525dc0854112c9ff55"}
Sep 14 07:49:48.541575 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.541527    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8hbxw" podStartSLOduration=2.7470007990000003 podStartE2EDuration="4.541510191s" totalImagesPullingTime="1.721441481s" totalInitContainerRuntime="73.067911ms" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:48.540769513 +0000 UTC m=+109.169657140" watchObservedRunningTime="2026-09-14 07:49:48.541510191 +0000 UTC m=+109.170397895"
Sep 14 07:49:48.765877 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.765802    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5b6cdf8787-kxws6"]
Sep 14 07:49:48.767883 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.767863    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:48.770875 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.770835    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 14 07:49:48.770875 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.770868    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 14 07:49:48.771051 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.770842    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bmh16948het7m\""
Sep 14 07:49:48.771051 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.770921    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 14 07:49:48.771051 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.770933    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 14 07:49:48.771051 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.770952    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-nnw5d\""
Sep 14 07:49:48.778518 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.778498    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b6cdf8787-kxws6"]
Sep 14 07:49:48.914490 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.914448    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a4d24038-849b-4139-8f73-730159f4421e-metrics-server-audit-profiles\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:48.914490 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.914498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a4d24038-849b-4139-8f73-730159f4421e-audit-log\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:48.914698 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.914545    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws5cw\" (UniqueName: \"kubernetes.io/projected/a4d24038-849b-4139-8f73-730159f4421e-kube-api-access-ws5cw\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:48.914698 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.914565    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4d24038-849b-4139-8f73-730159f4421e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:48.914698 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.914585    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a4d24038-849b-4139-8f73-730159f4421e-secret-metrics-server-client-certs\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:48.914698 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.914608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4d24038-849b-4139-8f73-730159f4421e-client-ca-bundle\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:48.914698 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:48.914668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a4d24038-849b-4139-8f73-730159f4421e-secret-metrics-server-tls\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.015863 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.015826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a4d24038-849b-4139-8f73-730159f4421e-metrics-server-audit-profiles\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.016053 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.015872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a4d24038-849b-4139-8f73-730159f4421e-audit-log\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.016053 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.015917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws5cw\" (UniqueName: \"kubernetes.io/projected/a4d24038-849b-4139-8f73-730159f4421e-kube-api-access-ws5cw\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.016126 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.016040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4d24038-849b-4139-8f73-730159f4421e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.016126 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.016090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a4d24038-849b-4139-8f73-730159f4421e-secret-metrics-server-client-certs\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.016126 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.016117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4d24038-849b-4139-8f73-730159f4421e-client-ca-bundle\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.016258 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.016170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a4d24038-849b-4139-8f73-730159f4421e-secret-metrics-server-tls\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.016384 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.016364    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a4d24038-849b-4139-8f73-730159f4421e-audit-log\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.016802 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.016777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4d24038-849b-4139-8f73-730159f4421e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.016928 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.016877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a4d24038-849b-4139-8f73-730159f4421e-metrics-server-audit-profiles\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.018720 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.018696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a4d24038-849b-4139-8f73-730159f4421e-secret-metrics-server-client-certs\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.018819 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.018790    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4d24038-849b-4139-8f73-730159f4421e-client-ca-bundle\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.018895 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.018864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a4d24038-849b-4139-8f73-730159f4421e-secret-metrics-server-tls\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.029022 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.029003    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws5cw\" (UniqueName: \"kubernetes.io/projected/a4d24038-849b-4139-8f73-730159f4421e-kube-api-access-ws5cw\") pod \"metrics-server-5b6cdf8787-kxws6\" (UID: \"a4d24038-849b-4139-8f73-730159f4421e\") " pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.077896 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.077870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:49:49.303827 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.303752    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9"]
Sep 14 07:49:49.306294 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.306278    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9"
Sep 14 07:49:49.313301 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.313280    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8xx5d\""
Sep 14 07:49:49.313400 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.313324    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 14 07:49:49.341840 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.341816    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9"]
Sep 14 07:49:49.360995 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.360973    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b6cdf8787-kxws6"]
Sep 14 07:49:49.363438 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:49.363416    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d24038_849b_4139_8f73_730159f4421e.slice/crio-4e06a304cc3b85dc54037456c499e3f1d086158d7ed5f877687c051657ff68ef WatchSource:0}: Error finding container 4e06a304cc3b85dc54037456c499e3f1d086158d7ed5f877687c051657ff68ef: Status 404 returned error can't find the container with id 4e06a304cc3b85dc54037456c499e3f1d086158d7ed5f877687c051657ff68ef
Sep 14 07:49:49.419456 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.419430    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz4ts\" (UniqueName: \"kubernetes.io/projected/172d736a-dd5b-47a8-b6f1-ea0b4c0af7af-kube-api-access-cz4ts\") pod \"monitoring-plugin-5d85778b9d-kvjf9\" (UID: \"172d736a-dd5b-47a8-b6f1-ea0b4c0af7af\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9"
Sep 14 07:49:49.419715 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.419471    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/172d736a-dd5b-47a8-b6f1-ea0b4c0af7af-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-kvjf9\" (UID: \"172d736a-dd5b-47a8-b6f1-ea0b4c0af7af\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9"
Sep 14 07:49:49.496439 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.496396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6" event={"ID":"a4d24038-849b-4139-8f73-730159f4421e","Type":"ContainerStarted","Data":"4e06a304cc3b85dc54037456c499e3f1d086158d7ed5f877687c051657ff68ef"}
Sep 14 07:49:49.520088 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.520060    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz4ts\" (UniqueName: \"kubernetes.io/projected/172d736a-dd5b-47a8-b6f1-ea0b4c0af7af-kube-api-access-cz4ts\") pod \"monitoring-plugin-5d85778b9d-kvjf9\" (UID: \"172d736a-dd5b-47a8-b6f1-ea0b4c0af7af\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9"
Sep 14 07:49:49.520216 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.520094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/172d736a-dd5b-47a8-b6f1-ea0b4c0af7af-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-kvjf9\" (UID: \"172d736a-dd5b-47a8-b6f1-ea0b4c0af7af\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9"
Sep 14 07:49:49.522593 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.522573    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/172d736a-dd5b-47a8-b6f1-ea0b4c0af7af-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-kvjf9\" (UID: \"172d736a-dd5b-47a8-b6f1-ea0b4c0af7af\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9"
Sep 14 07:49:49.528885 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.528859    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz4ts\" (UniqueName: \"kubernetes.io/projected/172d736a-dd5b-47a8-b6f1-ea0b4c0af7af-kube-api-access-cz4ts\") pod \"monitoring-plugin-5d85778b9d-kvjf9\" (UID: \"172d736a-dd5b-47a8-b6f1-ea0b4c0af7af\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9"
Sep 14 07:49:49.615059 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.615031    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9"
Sep 14 07:49:49.885225 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:49.885202    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9"]
Sep 14 07:49:49.886519 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:49.886490    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod172d736a_dd5b_47a8_b6f1_ea0b4c0af7af.slice/crio-d2d5d3e3cf5f05443980bf901b052cd829c1a787017f1e33627d1c893468207a WatchSource:0}: Error finding container d2d5d3e3cf5f05443980bf901b052cd829c1a787017f1e33627d1c893468207a: Status 404 returned error can't find the container with id d2d5d3e3cf5f05443980bf901b052cd829c1a787017f1e33627d1c893468207a
Sep 14 07:49:50.236725 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.236625    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-689d5c5446-9f6rw"]
Sep 14 07:49:50.240123 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.240097    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.243133 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.243109    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 14 07:49:50.243429 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.243386    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dfmfg\""
Sep 14 07:49:50.243543 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.243437    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 14 07:49:50.243543 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.243454    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 14 07:49:50.243543 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.243500    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 14 07:49:50.243842 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.243820    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 14 07:49:50.244140 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.243982    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 14 07:49:50.244140 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.244025    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 14 07:49:50.247742 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.247682    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 14 07:49:50.251294 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.251272    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689d5c5446-9f6rw"]
Sep 14 07:49:50.429235 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.429204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-service-ca\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.429235 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.429239    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-console-config\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.429438 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.429262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-oauth-serving-cert\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.429438 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.429349    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-trusted-ca-bundle\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.429438 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.429382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2glq\" (UniqueName: \"kubernetes.io/projected/798c518d-5123-4d0c-a746-2f444ebde33e-kube-api-access-g2glq\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.429539 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.429449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/798c518d-5123-4d0c-a746-2f444ebde33e-console-serving-cert\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.429539 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.429467    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/798c518d-5123-4d0c-a746-2f444ebde33e-console-oauth-config\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.502781 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.502615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9" event={"ID":"172d736a-dd5b-47a8-b6f1-ea0b4c0af7af","Type":"ContainerStarted","Data":"d2d5d3e3cf5f05443980bf901b052cd829c1a787017f1e33627d1c893468207a"}
Sep 14 07:49:50.530141 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.530111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-service-ca\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.530366 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.530148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-console-config\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.530366 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.530173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-oauth-serving-cert\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.530366 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.530219    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-trusted-ca-bundle\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.530366 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.530235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2glq\" (UniqueName: \"kubernetes.io/projected/798c518d-5123-4d0c-a746-2f444ebde33e-kube-api-access-g2glq\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.530656 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.530617    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/798c518d-5123-4d0c-a746-2f444ebde33e-console-serving-cert\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.530749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.530673    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/798c518d-5123-4d0c-a746-2f444ebde33e-console-oauth-config\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.530976 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.530942    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-service-ca\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.530976 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.530964    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-console-config\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.531445 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.531424    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-trusted-ca-bundle\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.531652 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.531595    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-oauth-serving-cert\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.533683 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.533661    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/798c518d-5123-4d0c-a746-2f444ebde33e-console-serving-cert\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.533782 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.533739    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/798c518d-5123-4d0c-a746-2f444ebde33e-console-oauth-config\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.544926 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.544906    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2glq\" (UniqueName: \"kubernetes.io/projected/798c518d-5123-4d0c-a746-2f444ebde33e-kube-api-access-g2glq\") pod \"console-689d5c5446-9f6rw\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") " pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:50.551702 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:50.551679    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:49:51.146947 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:51.146916    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689d5c5446-9f6rw"]
Sep 14 07:49:51.406456 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:49:51.406382    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod798c518d_5123_4d0c_a746_2f444ebde33e.slice/crio-0dfdcc7c7acf3cc4ce6a86a48c4409681a761f8ed8639a9983b08568d7e5b1a0 WatchSource:0}: Error finding container 0dfdcc7c7acf3cc4ce6a86a48c4409681a761f8ed8639a9983b08568d7e5b1a0: Status 404 returned error can't find the container with id 0dfdcc7c7acf3cc4ce6a86a48c4409681a761f8ed8639a9983b08568d7e5b1a0
Sep 14 07:49:51.510562 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:51.510457    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6" event={"ID":"a4d24038-849b-4139-8f73-730159f4421e","Type":"ContainerStarted","Data":"77c1d51784b05a0b8e10c3ae8e300db363bf9042984f475b6cd9b113ea7ddaa4"}
Sep 14 07:49:51.512451 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:51.512425    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689d5c5446-9f6rw" event={"ID":"798c518d-5123-4d0c-a746-2f444ebde33e","Type":"ContainerStarted","Data":"0dfdcc7c7acf3cc4ce6a86a48c4409681a761f8ed8639a9983b08568d7e5b1a0"}
Sep 14 07:49:51.530272 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:51.530223    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6" podStartSLOduration=1.96138283 podStartE2EDuration="3.530206749s" totalImagesPullingTime="1.568823919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:51.52826332 +0000 UTC m=+112.157150946" watchObservedRunningTime="2026-09-14 07:49:51.530206749 +0000 UTC m=+112.159094376"
Sep 14 07:49:52.518873 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:52.518831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9" event={"ID":"172d736a-dd5b-47a8-b6f1-ea0b4c0af7af","Type":"ContainerStarted","Data":"c9a23dba39ae4499a35c30d3a2f3faa36d9ce0e4e0f2c57c7a9ede30c8848609"}
Sep 14 07:49:52.519320 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:52.519064    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9"
Sep 14 07:49:52.524427 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:52.524402    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9"
Sep 14 07:49:52.536510 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:52.536467    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-kvjf9" podStartSLOduration=1.972949034 podStartE2EDuration="3.536449172s" totalImagesPullingTime="1.563500138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:52.535891369 +0000 UTC m=+113.164779019" watchObservedRunningTime="2026-09-14 07:49:52.536449172 +0000 UTC m=+113.165336800"
Sep 14 07:49:54.529188 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:54.529150    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689d5c5446-9f6rw" event={"ID":"798c518d-5123-4d0c-a746-2f444ebde33e","Type":"ContainerStarted","Data":"4330f4693a9c2dc1630122ed2782ade1c6de726d55e187916159615495da11d5"}
Sep 14 07:49:54.549709 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:54.549668    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-689d5c5446-9f6rw" podStartSLOduration=1.9093896190000001 podStartE2EDuration="4.549654028s" totalImagesPullingTime="2.640264409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:54.549389965 +0000 UTC m=+115.178277589" watchObservedRunningTime="2026-09-14 07:49:54.549654028 +0000 UTC m=+115.178541645"
Sep 14 07:49:55.478833 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:49:55.478799    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689d5c5446-9f6rw"]
Sep 14 07:50:00.552727 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:00.552697    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:50:09.078305 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:09.078254    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:50:09.078723 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:09.078335    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:50:09.589829 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:09.589801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:50:09.592026 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:09.592001    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257-metrics-certs\") pod \"network-metrics-daemon-46qgp\" (UID: \"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257\") " pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:50:09.893514 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:09.893433    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bd4sd\""
Sep 14 07:50:09.901226 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:09.901210    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46qgp"
Sep 14 07:50:10.169537 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:10.169511    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-46qgp"]
Sep 14 07:50:10.171996 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:50:10.171969    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e3aa91_6cdf_4878_a9c1_12f7e5dfd257.slice/crio-152e2eea9613b6eed025602dbc9d7f1f5deff81933ff1dd389505255a31e4fa9 WatchSource:0}: Error finding container 152e2eea9613b6eed025602dbc9d7f1f5deff81933ff1dd389505255a31e4fa9: Status 404 returned error can't find the container with id 152e2eea9613b6eed025602dbc9d7f1f5deff81933ff1dd389505255a31e4fa9
Sep 14 07:50:10.592980 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:10.592946    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-46qgp" event={"ID":"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257","Type":"ContainerStarted","Data":"152e2eea9613b6eed025602dbc9d7f1f5deff81933ff1dd389505255a31e4fa9"}
Sep 14 07:50:11.598533 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:11.598489    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-46qgp" event={"ID":"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257","Type":"ContainerStarted","Data":"939113c3ee2dc0f9aaf3afadf88b5c0bdf2e2880ce33f2c15ca398ced284fbf3"}
Sep 14 07:50:11.598959 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:11.598543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-46qgp" event={"ID":"c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257","Type":"ContainerStarted","Data":"29a18de81532bd30c60bab598504055bd5b5a84e97bf0d7f365beb9089b64942"}
Sep 14 07:50:11.628729 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:11.628686    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-46qgp" podStartSLOduration=130.61479937 podStartE2EDuration="2m11.628669477s" totalImagesPullingTime="1.013870107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:11.628097653 +0000 UTC m=+132.256985298" watchObservedRunningTime="2026-09-14 07:50:11.628669477 +0000 UTC m=+132.257557100"
Sep 14 07:50:21.555786 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.555724    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-689d5c5446-9f6rw" podUID="798c518d-5123-4d0c-a746-2f444ebde33e" containerName="console" containerID="cri-o://4330f4693a9c2dc1630122ed2782ade1c6de726d55e187916159615495da11d5" gracePeriod=15
Sep 14 07:50:21.802040 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.802015    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:50:21.899003 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.898979    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/798c518d-5123-4d0c-a746-2f444ebde33e-console-serving-cert\") pod \"798c518d-5123-4d0c-a746-2f444ebde33e\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") "
Sep 14 07:50:21.899166 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.899014    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-service-ca\") pod \"798c518d-5123-4d0c-a746-2f444ebde33e\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") "
Sep 14 07:50:21.899166 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.899031    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-oauth-serving-cert\") pod \"798c518d-5123-4d0c-a746-2f444ebde33e\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") "
Sep 14 07:50:21.899166 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.899055    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2glq\" (UniqueName: \"kubernetes.io/projected/798c518d-5123-4d0c-a746-2f444ebde33e-kube-api-access-g2glq\") pod \"798c518d-5123-4d0c-a746-2f444ebde33e\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") "
Sep 14 07:50:21.899166 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.899122    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/798c518d-5123-4d0c-a746-2f444ebde33e-console-oauth-config\") pod \"798c518d-5123-4d0c-a746-2f444ebde33e\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") "
Sep 14 07:50:21.899379 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.899217    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-trusted-ca-bundle\") pod \"798c518d-5123-4d0c-a746-2f444ebde33e\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") "
Sep 14 07:50:21.899379 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.899251    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-console-config\") pod \"798c518d-5123-4d0c-a746-2f444ebde33e\" (UID: \"798c518d-5123-4d0c-a746-2f444ebde33e\") "
Sep 14 07:50:21.899537 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.899507    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-service-ca" (OuterVolumeSpecName: "service-ca") pod "798c518d-5123-4d0c-a746-2f444ebde33e" (UID: "798c518d-5123-4d0c-a746-2f444ebde33e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:21.899594 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.899512    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "798c518d-5123-4d0c-a746-2f444ebde33e" (UID: "798c518d-5123-4d0c-a746-2f444ebde33e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:21.899730 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.899688    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "798c518d-5123-4d0c-a746-2f444ebde33e" (UID: "798c518d-5123-4d0c-a746-2f444ebde33e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:21.899906 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.899797    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-console-config" (OuterVolumeSpecName: "console-config") pod "798c518d-5123-4d0c-a746-2f444ebde33e" (UID: "798c518d-5123-4d0c-a746-2f444ebde33e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:21.901287 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.901268    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/798c518d-5123-4d0c-a746-2f444ebde33e-kube-api-access-g2glq" (OuterVolumeSpecName: "kube-api-access-g2glq") pod "798c518d-5123-4d0c-a746-2f444ebde33e" (UID: "798c518d-5123-4d0c-a746-2f444ebde33e"). InnerVolumeSpecName "kube-api-access-g2glq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:50:21.901575 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.901549    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/798c518d-5123-4d0c-a746-2f444ebde33e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "798c518d-5123-4d0c-a746-2f444ebde33e" (UID: "798c518d-5123-4d0c-a746-2f444ebde33e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:50:21.901625 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.901561    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/798c518d-5123-4d0c-a746-2f444ebde33e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "798c518d-5123-4d0c-a746-2f444ebde33e" (UID: "798c518d-5123-4d0c-a746-2f444ebde33e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:50:21.999962 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.999932    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/798c518d-5123-4d0c-a746-2f444ebde33e-console-serving-cert\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:50:21.999962 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.999957    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-service-ca\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:50:21.999962 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.999966    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-oauth-serving-cert\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:50:22.000146 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.999976    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2glq\" (UniqueName: \"kubernetes.io/projected/798c518d-5123-4d0c-a746-2f444ebde33e-kube-api-access-g2glq\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:50:22.000146 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.999984    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/798c518d-5123-4d0c-a746-2f444ebde33e-console-oauth-config\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:50:22.000146 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:21.999993    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-trusted-ca-bundle\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:50:22.000146 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:22.000000    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/798c518d-5123-4d0c-a746-2f444ebde33e-console-config\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:50:22.641998 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:22.641963    2563 generic.go:356] "Generic (PLEG): container finished" podID="798c518d-5123-4d0c-a746-2f444ebde33e" containerID="4330f4693a9c2dc1630122ed2782ade1c6de726d55e187916159615495da11d5" exitCode=0
Sep 14 07:50:22.642376 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:22.642012    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689d5c5446-9f6rw"
Sep 14 07:50:22.642376 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:22.642049    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689d5c5446-9f6rw" event={"ID":"798c518d-5123-4d0c-a746-2f444ebde33e","Type":"ContainerDied","Data":"4330f4693a9c2dc1630122ed2782ade1c6de726d55e187916159615495da11d5"}
Sep 14 07:50:22.642376 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:22.642086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689d5c5446-9f6rw" event={"ID":"798c518d-5123-4d0c-a746-2f444ebde33e","Type":"ContainerDied","Data":"0dfdcc7c7acf3cc4ce6a86a48c4409681a761f8ed8639a9983b08568d7e5b1a0"}
Sep 14 07:50:22.642376 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:22.642132    2563 scope.go:117] "RemoveContainer" containerID="4330f4693a9c2dc1630122ed2782ade1c6de726d55e187916159615495da11d5"
Sep 14 07:50:22.650779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:22.650761    2563 scope.go:117] "RemoveContainer" containerID="4330f4693a9c2dc1630122ed2782ade1c6de726d55e187916159615495da11d5"
Sep 14 07:50:22.651019 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:50:22.651001    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4330f4693a9c2dc1630122ed2782ade1c6de726d55e187916159615495da11d5\": container with ID starting with 4330f4693a9c2dc1630122ed2782ade1c6de726d55e187916159615495da11d5 not found: ID does not exist" containerID="4330f4693a9c2dc1630122ed2782ade1c6de726d55e187916159615495da11d5"
Sep 14 07:50:22.651061 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:22.651027    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4330f4693a9c2dc1630122ed2782ade1c6de726d55e187916159615495da11d5"} err="failed to get container status \"4330f4693a9c2dc1630122ed2782ade1c6de726d55e187916159615495da11d5\": rpc error: code = NotFound desc = could not find container \"4330f4693a9c2dc1630122ed2782ade1c6de726d55e187916159615495da11d5\": container with ID starting with 4330f4693a9c2dc1630122ed2782ade1c6de726d55e187916159615495da11d5 not found: ID does not exist"
Sep 14 07:50:22.663905 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:22.663877    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689d5c5446-9f6rw"]
Sep 14 07:50:22.667814 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:22.667795    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-689d5c5446-9f6rw"]
Sep 14 07:50:23.885200 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:23.885165    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="798c518d-5123-4d0c-a746-2f444ebde33e" path="/var/lib/kubelet/pods/798c518d-5123-4d0c-a746-2f444ebde33e/volumes"
Sep 14 07:50:24.651267 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:24.651231    2563 generic.go:356] "Generic (PLEG): container finished" podID="4e45fd58-3953-4aca-89d6-d8facaaf9a28" containerID="8f06332f0efcd0bc7ce6c20770bf16a0cfb7aa967781ba1818c5a785db8f4608" exitCode=0
Sep 14 07:50:24.651492 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:24.651273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh" event={"ID":"4e45fd58-3953-4aca-89d6-d8facaaf9a28","Type":"ContainerDied","Data":"8f06332f0efcd0bc7ce6c20770bf16a0cfb7aa967781ba1818c5a785db8f4608"}
Sep 14 07:50:24.651720 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:24.651702    2563 scope.go:117] "RemoveContainer" containerID="8f06332f0efcd0bc7ce6c20770bf16a0cfb7aa967781ba1818c5a785db8f4608"
Sep 14 07:50:25.657481 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:25.657447    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-hb2jh" event={"ID":"4e45fd58-3953-4aca-89d6-d8facaaf9a28","Type":"ContainerStarted","Data":"b23b4920ad6eabf01ae74444656dee549040545185858893f90ef66183ac4bce"}
Sep 14 07:50:29.083801 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:29.083770    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:50:29.087570 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:50:29.087546    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5b6cdf8787-kxws6"
Sep 14 07:52:05.067780 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.067747    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"]
Sep 14 07:52:05.068214 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.068019    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="798c518d-5123-4d0c-a746-2f444ebde33e" containerName="console"
Sep 14 07:52:05.068214 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.068029    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="798c518d-5123-4d0c-a746-2f444ebde33e" containerName="console"
Sep 14 07:52:05.068214 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.068092    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="798c518d-5123-4d0c-a746-2f444ebde33e" containerName="console"
Sep 14 07:52:05.071241 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.071225    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"
Sep 14 07:52:05.075829 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.075799    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 07:52:05.076254 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.076230    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 07:52:05.076374 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.076255    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-q6vxq\""
Sep 14 07:52:05.097630 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.097595    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"]
Sep 14 07:52:05.102748 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.102721    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e358dd5-f416-435a-949e-2ae35b139f69-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn\" (UID: \"4e358dd5-f416-435a-949e-2ae35b139f69\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"
Sep 14 07:52:05.102875 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.102776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e358dd5-f416-435a-949e-2ae35b139f69-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn\" (UID: \"4e358dd5-f416-435a-949e-2ae35b139f69\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"
Sep 14 07:52:05.102875 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.102853    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjbzt\" (UniqueName: \"kubernetes.io/projected/4e358dd5-f416-435a-949e-2ae35b139f69-kube-api-access-kjbzt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn\" (UID: \"4e358dd5-f416-435a-949e-2ae35b139f69\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"
Sep 14 07:52:05.203609 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.203569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e358dd5-f416-435a-949e-2ae35b139f69-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn\" (UID: \"4e358dd5-f416-435a-949e-2ae35b139f69\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"
Sep 14 07:52:05.203812 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.203626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e358dd5-f416-435a-949e-2ae35b139f69-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn\" (UID: \"4e358dd5-f416-435a-949e-2ae35b139f69\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"
Sep 14 07:52:05.203812 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.203684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjbzt\" (UniqueName: \"kubernetes.io/projected/4e358dd5-f416-435a-949e-2ae35b139f69-kube-api-access-kjbzt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn\" (UID: \"4e358dd5-f416-435a-949e-2ae35b139f69\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"
Sep 14 07:52:05.203980 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.203960    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e358dd5-f416-435a-949e-2ae35b139f69-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn\" (UID: \"4e358dd5-f416-435a-949e-2ae35b139f69\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"
Sep 14 07:52:05.204027 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.203996    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e358dd5-f416-435a-949e-2ae35b139f69-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn\" (UID: \"4e358dd5-f416-435a-949e-2ae35b139f69\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"
Sep 14 07:52:05.213461 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.213432    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjbzt\" (UniqueName: \"kubernetes.io/projected/4e358dd5-f416-435a-949e-2ae35b139f69-kube-api-access-kjbzt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn\" (UID: \"4e358dd5-f416-435a-949e-2ae35b139f69\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"
Sep 14 07:52:05.379752 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.379715    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"
Sep 14 07:52:05.667419 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:05.667215    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"]
Sep 14 07:52:05.669907 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:52:05.669871    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e358dd5_f416_435a_949e_2ae35b139f69.slice/crio-063e3ee7d05cd1b9cda22bc89e372f68ccf337c3f8dfd70408df966c14368968 WatchSource:0}: Error finding container 063e3ee7d05cd1b9cda22bc89e372f68ccf337c3f8dfd70408df966c14368968: Status 404 returned error can't find the container with id 063e3ee7d05cd1b9cda22bc89e372f68ccf337c3f8dfd70408df966c14368968
Sep 14 07:52:06.018095 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:06.018008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn" event={"ID":"4e358dd5-f416-435a-949e-2ae35b139f69","Type":"ContainerStarted","Data":"063e3ee7d05cd1b9cda22bc89e372f68ccf337c3f8dfd70408df966c14368968"}
Sep 14 07:52:12.042850 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:12.042794    2563 generic.go:356] "Generic (PLEG): container finished" podID="4e358dd5-f416-435a-949e-2ae35b139f69" containerID="4daed90366b65f6c8fd6de81e644992291199f6d08df30da6181303f1a405d13" exitCode=0
Sep 14 07:52:12.042850 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:12.042840    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn" event={"ID":"4e358dd5-f416-435a-949e-2ae35b139f69","Type":"ContainerDied","Data":"4daed90366b65f6c8fd6de81e644992291199f6d08df30da6181303f1a405d13"}
Sep 14 07:52:18.066320 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:18.066287    2563 generic.go:356] "Generic (PLEG): container finished" podID="4e358dd5-f416-435a-949e-2ae35b139f69" containerID="1e76cc962b110b6f4af901b60a27e7900ef84029237ca9ffd76a84557325ab48" exitCode=0
Sep 14 07:52:18.066778 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:18.066373    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn" event={"ID":"4e358dd5-f416-435a-949e-2ae35b139f69","Type":"ContainerDied","Data":"1e76cc962b110b6f4af901b60a27e7900ef84029237ca9ffd76a84557325ab48"}
Sep 14 07:52:25.097857 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:25.097818    2563 generic.go:356] "Generic (PLEG): container finished" podID="4e358dd5-f416-435a-949e-2ae35b139f69" containerID="27a6fa3cbc8275e6621ae0ebbce36892ec3ef4b8f6814d0f3b4f1db7819ce7ed" exitCode=0
Sep 14 07:52:25.098225 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:25.097893    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn" event={"ID":"4e358dd5-f416-435a-949e-2ae35b139f69","Type":"ContainerDied","Data":"27a6fa3cbc8275e6621ae0ebbce36892ec3ef4b8f6814d0f3b4f1db7819ce7ed"}
Sep 14 07:52:26.223472 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:26.223449    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"
Sep 14 07:52:26.275941 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:26.275910    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjbzt\" (UniqueName: \"kubernetes.io/projected/4e358dd5-f416-435a-949e-2ae35b139f69-kube-api-access-kjbzt\") pod \"4e358dd5-f416-435a-949e-2ae35b139f69\" (UID: \"4e358dd5-f416-435a-949e-2ae35b139f69\") "
Sep 14 07:52:26.276100 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:26.275948    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e358dd5-f416-435a-949e-2ae35b139f69-util\") pod \"4e358dd5-f416-435a-949e-2ae35b139f69\" (UID: \"4e358dd5-f416-435a-949e-2ae35b139f69\") "
Sep 14 07:52:26.276100 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:26.275985    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e358dd5-f416-435a-949e-2ae35b139f69-bundle\") pod \"4e358dd5-f416-435a-949e-2ae35b139f69\" (UID: \"4e358dd5-f416-435a-949e-2ae35b139f69\") "
Sep 14 07:52:26.276634 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:26.276610    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e358dd5-f416-435a-949e-2ae35b139f69-bundle" (OuterVolumeSpecName: "bundle") pod "4e358dd5-f416-435a-949e-2ae35b139f69" (UID: "4e358dd5-f416-435a-949e-2ae35b139f69"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:52:26.278130 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:26.278095    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e358dd5-f416-435a-949e-2ae35b139f69-kube-api-access-kjbzt" (OuterVolumeSpecName: "kube-api-access-kjbzt") pod "4e358dd5-f416-435a-949e-2ae35b139f69" (UID: "4e358dd5-f416-435a-949e-2ae35b139f69"). InnerVolumeSpecName "kube-api-access-kjbzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:52:26.280476 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:26.280452    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e358dd5-f416-435a-949e-2ae35b139f69-util" (OuterVolumeSpecName: "util") pod "4e358dd5-f416-435a-949e-2ae35b139f69" (UID: "4e358dd5-f416-435a-949e-2ae35b139f69"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:52:26.376684 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:26.376597    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e358dd5-f416-435a-949e-2ae35b139f69-bundle\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:52:26.376684 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:26.376622    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjbzt\" (UniqueName: \"kubernetes.io/projected/4e358dd5-f416-435a-949e-2ae35b139f69-kube-api-access-kjbzt\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:52:26.376684 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:26.376635    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e358dd5-f416-435a-949e-2ae35b139f69-util\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:52:27.107590 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:27.107557    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn" event={"ID":"4e358dd5-f416-435a-949e-2ae35b139f69","Type":"ContainerDied","Data":"063e3ee7d05cd1b9cda22bc89e372f68ccf337c3f8dfd70408df966c14368968"}
Sep 14 07:52:27.107590 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:27.107577    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wr5bn"
Sep 14 07:52:27.107809 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:27.107608    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="063e3ee7d05cd1b9cda22bc89e372f68ccf337c3f8dfd70408df966c14368968"
Sep 14 07:52:32.794917 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.794883    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq"]
Sep 14 07:52:32.795289 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.795159    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e358dd5-f416-435a-949e-2ae35b139f69" containerName="util"
Sep 14 07:52:32.795289 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.795170    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e358dd5-f416-435a-949e-2ae35b139f69" containerName="util"
Sep 14 07:52:32.795289 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.795178    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e358dd5-f416-435a-949e-2ae35b139f69" containerName="pull"
Sep 14 07:52:32.795289 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.795185    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e358dd5-f416-435a-949e-2ae35b139f69" containerName="pull"
Sep 14 07:52:32.795289 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.795196    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e358dd5-f416-435a-949e-2ae35b139f69" containerName="extract"
Sep 14 07:52:32.795289 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.795201    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e358dd5-f416-435a-949e-2ae35b139f69" containerName="extract"
Sep 14 07:52:32.795289 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.795254    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e358dd5-f416-435a-949e-2ae35b139f69" containerName="extract"
Sep 14 07:52:32.798227 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.798211    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq"
Sep 14 07:52:32.800381 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.800359    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 14 07:52:32.800504 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.800421    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:52:32.800504 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.800443    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8hq4z\""
Sep 14 07:52:32.809458 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.809434    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq"]
Sep 14 07:52:32.926618 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.926573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z4sh\" (UniqueName: \"kubernetes.io/projected/683fb8ab-231d-4a8b-9613-1a47dd19ae4b-kube-api-access-5z4sh\") pod \"cert-manager-operator-controller-manager-55c444bf86-dc7zq\" (UID: \"683fb8ab-231d-4a8b-9613-1a47dd19ae4b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq"
Sep 14 07:52:32.926817 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:32.926663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/683fb8ab-231d-4a8b-9613-1a47dd19ae4b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dc7zq\" (UID: \"683fb8ab-231d-4a8b-9613-1a47dd19ae4b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq"
Sep 14 07:52:33.027051 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:33.027003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z4sh\" (UniqueName: \"kubernetes.io/projected/683fb8ab-231d-4a8b-9613-1a47dd19ae4b-kube-api-access-5z4sh\") pod \"cert-manager-operator-controller-manager-55c444bf86-dc7zq\" (UID: \"683fb8ab-231d-4a8b-9613-1a47dd19ae4b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq"
Sep 14 07:52:33.027051 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:33.027061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/683fb8ab-231d-4a8b-9613-1a47dd19ae4b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dc7zq\" (UID: \"683fb8ab-231d-4a8b-9613-1a47dd19ae4b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq"
Sep 14 07:52:33.027433 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:33.027415    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/683fb8ab-231d-4a8b-9613-1a47dd19ae4b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dc7zq\" (UID: \"683fb8ab-231d-4a8b-9613-1a47dd19ae4b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq"
Sep 14 07:52:33.042726 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:33.042702    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z4sh\" (UniqueName: \"kubernetes.io/projected/683fb8ab-231d-4a8b-9613-1a47dd19ae4b-kube-api-access-5z4sh\") pod \"cert-manager-operator-controller-manager-55c444bf86-dc7zq\" (UID: \"683fb8ab-231d-4a8b-9613-1a47dd19ae4b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq"
Sep 14 07:52:33.107225 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:33.107195    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq"
Sep 14 07:52:33.382729 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:33.382628    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq"]
Sep 14 07:52:33.388393 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:52:33.388361    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683fb8ab_231d_4a8b_9613_1a47dd19ae4b.slice/crio-21e3c36fa23dbcc9e93ec292b62d36d762d29d7eea297ecb05b44d2274e7f779 WatchSource:0}: Error finding container 21e3c36fa23dbcc9e93ec292b62d36d762d29d7eea297ecb05b44d2274e7f779: Status 404 returned error can't find the container with id 21e3c36fa23dbcc9e93ec292b62d36d762d29d7eea297ecb05b44d2274e7f779
Sep 14 07:52:34.134186 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:34.134150    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq" event={"ID":"683fb8ab-231d-4a8b-9613-1a47dd19ae4b","Type":"ContainerStarted","Data":"21e3c36fa23dbcc9e93ec292b62d36d762d29d7eea297ecb05b44d2274e7f779"}
Sep 14 07:52:36.146295 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:36.146261    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq" event={"ID":"683fb8ab-231d-4a8b-9613-1a47dd19ae4b","Type":"ContainerStarted","Data":"82617a4c02565f8535a5a9180cb790749553d797ce3c7289eb093396c270b65d"}
Sep 14 07:52:36.167974 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:36.167922    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dc7zq" podStartSLOduration=2.482387364 podStartE2EDuration="4.167906772s" totalImagesPullingTime="1.685519408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:52:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:52:36.166586045 +0000 UTC m=+276.795473672" watchObservedRunningTime="2026-09-14 07:52:36.167906772 +0000 UTC m=+276.796794400"
Sep 14 07:52:37.315827 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.315790    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"]
Sep 14 07:52:37.319139 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.319116    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"
Sep 14 07:52:37.321345 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.321324    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 07:52:37.321878 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.321856    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-q6vxq\""
Sep 14 07:52:37.321983 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.321862    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 07:52:37.328488 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.328465    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"]
Sep 14 07:52:37.470827 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.470790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8e5b06d-1657-4271-8cbf-ee645dbd2027-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db\" (UID: \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"
Sep 14 07:52:37.471025 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.470917    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8e5b06d-1657-4271-8cbf-ee645dbd2027-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db\" (UID: \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"
Sep 14 07:52:37.471025 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.470965    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vltzm\" (UniqueName: \"kubernetes.io/projected/e8e5b06d-1657-4271-8cbf-ee645dbd2027-kube-api-access-vltzm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db\" (UID: \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"
Sep 14 07:52:37.571920 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.571825    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8e5b06d-1657-4271-8cbf-ee645dbd2027-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db\" (UID: \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"
Sep 14 07:52:37.571920 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.571917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8e5b06d-1657-4271-8cbf-ee645dbd2027-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db\" (UID: \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"
Sep 14 07:52:37.572120 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.571944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vltzm\" (UniqueName: \"kubernetes.io/projected/e8e5b06d-1657-4271-8cbf-ee645dbd2027-kube-api-access-vltzm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db\" (UID: \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"
Sep 14 07:52:37.572227 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.572205    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8e5b06d-1657-4271-8cbf-ee645dbd2027-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db\" (UID: \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"
Sep 14 07:52:37.572302 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.572280    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8e5b06d-1657-4271-8cbf-ee645dbd2027-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db\" (UID: \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"
Sep 14 07:52:37.583742 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.583720    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vltzm\" (UniqueName: \"kubernetes.io/projected/e8e5b06d-1657-4271-8cbf-ee645dbd2027-kube-api-access-vltzm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db\" (UID: \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"
Sep 14 07:52:37.628808 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.628768    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"
Sep 14 07:52:37.914773 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:37.914742    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"]
Sep 14 07:52:37.917701 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:52:37.917666    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8e5b06d_1657_4271_8cbf_ee645dbd2027.slice/crio-757cec2c5df5e73d9c0e5f3bc5bdc4cf93ca86e5fed2db9c913593c19e8b7aac WatchSource:0}: Error finding container 757cec2c5df5e73d9c0e5f3bc5bdc4cf93ca86e5fed2db9c913593c19e8b7aac: Status 404 returned error can't find the container with id 757cec2c5df5e73d9c0e5f3bc5bdc4cf93ca86e5fed2db9c913593c19e8b7aac
Sep 14 07:52:38.155672 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:38.155627    2563 generic.go:356] "Generic (PLEG): container finished" podID="e8e5b06d-1657-4271-8cbf-ee645dbd2027" containerID="8363d738fb10be9a7c38254f199af7687ea7a6a0a3fe115ee9861fe6f60240cb" exitCode=0
Sep 14 07:52:38.155672 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:38.155675    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db" event={"ID":"e8e5b06d-1657-4271-8cbf-ee645dbd2027","Type":"ContainerDied","Data":"8363d738fb10be9a7c38254f199af7687ea7a6a0a3fe115ee9861fe6f60240cb"}
Sep 14 07:52:38.155882 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:38.155730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db" event={"ID":"e8e5b06d-1657-4271-8cbf-ee645dbd2027","Type":"ContainerStarted","Data":"757cec2c5df5e73d9c0e5f3bc5bdc4cf93ca86e5fed2db9c913593c19e8b7aac"}
Sep 14 07:52:40.171304 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:40.171268    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db" event={"ID":"e8e5b06d-1657-4271-8cbf-ee645dbd2027","Type":"ContainerStarted","Data":"a2a6c9dd6c9059c88061513184b30ed31b6f951832120babf9ea0d6e93e27c03"}
Sep 14 07:52:41.177368 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:41.177336    2563 generic.go:356] "Generic (PLEG): container finished" podID="e8e5b06d-1657-4271-8cbf-ee645dbd2027" containerID="a2a6c9dd6c9059c88061513184b30ed31b6f951832120babf9ea0d6e93e27c03" exitCode=0
Sep 14 07:52:41.177779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:41.177423    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db" event={"ID":"e8e5b06d-1657-4271-8cbf-ee645dbd2027","Type":"ContainerDied","Data":"a2a6c9dd6c9059c88061513184b30ed31b6f951832120babf9ea0d6e93e27c03"}
Sep 14 07:52:42.183493 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:42.183458    2563 generic.go:356] "Generic (PLEG): container finished" podID="e8e5b06d-1657-4271-8cbf-ee645dbd2027" containerID="51be86f4703cced541f8c37eb13d3919eb8f5e547b2699c180740940a58bee88" exitCode=0
Sep 14 07:52:42.183950 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:42.183528    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db" event={"ID":"e8e5b06d-1657-4271-8cbf-ee645dbd2027","Type":"ContainerDied","Data":"51be86f4703cced541f8c37eb13d3919eb8f5e547b2699c180740940a58bee88"}
Sep 14 07:52:43.316774 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:43.316745    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"
Sep 14 07:52:43.417735 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:43.417698    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8e5b06d-1657-4271-8cbf-ee645dbd2027-bundle\") pod \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\" (UID: \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\") "
Sep 14 07:52:43.417918 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:43.417765    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vltzm\" (UniqueName: \"kubernetes.io/projected/e8e5b06d-1657-4271-8cbf-ee645dbd2027-kube-api-access-vltzm\") pod \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\" (UID: \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\") "
Sep 14 07:52:43.417918 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:43.417796    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8e5b06d-1657-4271-8cbf-ee645dbd2027-util\") pod \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\" (UID: \"e8e5b06d-1657-4271-8cbf-ee645dbd2027\") "
Sep 14 07:52:43.418176 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:43.418152    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8e5b06d-1657-4271-8cbf-ee645dbd2027-bundle" (OuterVolumeSpecName: "bundle") pod "e8e5b06d-1657-4271-8cbf-ee645dbd2027" (UID: "e8e5b06d-1657-4271-8cbf-ee645dbd2027"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:52:43.419919 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:43.419897    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8e5b06d-1657-4271-8cbf-ee645dbd2027-kube-api-access-vltzm" (OuterVolumeSpecName: "kube-api-access-vltzm") pod "e8e5b06d-1657-4271-8cbf-ee645dbd2027" (UID: "e8e5b06d-1657-4271-8cbf-ee645dbd2027"). InnerVolumeSpecName "kube-api-access-vltzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:52:43.423292 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:43.423269    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8e5b06d-1657-4271-8cbf-ee645dbd2027-util" (OuterVolumeSpecName: "util") pod "e8e5b06d-1657-4271-8cbf-ee645dbd2027" (UID: "e8e5b06d-1657-4271-8cbf-ee645dbd2027"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:52:43.518746 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:43.518671    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vltzm\" (UniqueName: \"kubernetes.io/projected/e8e5b06d-1657-4271-8cbf-ee645dbd2027-kube-api-access-vltzm\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:52:43.518746 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:43.518705    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8e5b06d-1657-4271-8cbf-ee645dbd2027-util\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:52:43.518746 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:43.518717    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8e5b06d-1657-4271-8cbf-ee645dbd2027-bundle\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:52:44.194197 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:44.194114    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db"
Sep 14 07:52:44.194197 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:44.194132    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdv2db" event={"ID":"e8e5b06d-1657-4271-8cbf-ee645dbd2027","Type":"ContainerDied","Data":"757cec2c5df5e73d9c0e5f3bc5bdc4cf93ca86e5fed2db9c913593c19e8b7aac"}
Sep 14 07:52:44.194197 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:44.194172    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="757cec2c5df5e73d9c0e5f3bc5bdc4cf93ca86e5fed2db9c913593c19e8b7aac"
Sep 14 07:52:51.218517 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.218484    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh"]
Sep 14 07:52:51.218931 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.218793    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8e5b06d-1657-4271-8cbf-ee645dbd2027" containerName="pull"
Sep 14 07:52:51.218931 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.218809    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8e5b06d-1657-4271-8cbf-ee645dbd2027" containerName="pull"
Sep 14 07:52:51.218931 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.218825    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8e5b06d-1657-4271-8cbf-ee645dbd2027" containerName="util"
Sep 14 07:52:51.218931 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.218832    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8e5b06d-1657-4271-8cbf-ee645dbd2027" containerName="util"
Sep 14 07:52:51.218931 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.218845    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8e5b06d-1657-4271-8cbf-ee645dbd2027" containerName="extract"
Sep 14 07:52:51.218931 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.218853    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8e5b06d-1657-4271-8cbf-ee645dbd2027" containerName="extract"
Sep 14 07:52:51.218931 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.218913    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8e5b06d-1657-4271-8cbf-ee645dbd2027" containerName="extract"
Sep 14 07:52:51.226378 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.226349    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh"
Sep 14 07:52:51.228796 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.228777    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 14 07:52:51.229349 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.229335    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:52:51.229399 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.229351    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-swhkn\""
Sep 14 07:52:51.235751 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.235726    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh"]
Sep 14 07:52:51.385331 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.385299    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6351179-d647-424e-a7ba-87389b3df3fe-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vmkvh\" (UID: \"b6351179-d647-424e-a7ba-87389b3df3fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh"
Sep 14 07:52:51.385331 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.385334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtdmg\" (UniqueName: \"kubernetes.io/projected/b6351179-d647-424e-a7ba-87389b3df3fe-kube-api-access-jtdmg\") pod \"openshift-lws-operator-fd8ccff4c-vmkvh\" (UID: \"b6351179-d647-424e-a7ba-87389b3df3fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh"
Sep 14 07:52:51.486661 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.486566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6351179-d647-424e-a7ba-87389b3df3fe-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vmkvh\" (UID: \"b6351179-d647-424e-a7ba-87389b3df3fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh"
Sep 14 07:52:51.486661 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.486603    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtdmg\" (UniqueName: \"kubernetes.io/projected/b6351179-d647-424e-a7ba-87389b3df3fe-kube-api-access-jtdmg\") pod \"openshift-lws-operator-fd8ccff4c-vmkvh\" (UID: \"b6351179-d647-424e-a7ba-87389b3df3fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh"
Sep 14 07:52:51.487049 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.487024    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6351179-d647-424e-a7ba-87389b3df3fe-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vmkvh\" (UID: \"b6351179-d647-424e-a7ba-87389b3df3fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh"
Sep 14 07:52:51.497014 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.496970    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtdmg\" (UniqueName: \"kubernetes.io/projected/b6351179-d647-424e-a7ba-87389b3df3fe-kube-api-access-jtdmg\") pod \"openshift-lws-operator-fd8ccff4c-vmkvh\" (UID: \"b6351179-d647-424e-a7ba-87389b3df3fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh"
Sep 14 07:52:51.543195 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.543161    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh"
Sep 14 07:52:51.814730 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:51.814704    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh"]
Sep 14 07:52:51.817044 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:52:51.817010    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6351179_d647_424e_a7ba_87389b3df3fe.slice/crio-d2b7eda8126d239bf1c24a08f4fd8d5835c285ebfcc87174a692ee4ecc06c1ae WatchSource:0}: Error finding container d2b7eda8126d239bf1c24a08f4fd8d5835c285ebfcc87174a692ee4ecc06c1ae: Status 404 returned error can't find the container with id d2b7eda8126d239bf1c24a08f4fd8d5835c285ebfcc87174a692ee4ecc06c1ae
Sep 14 07:52:52.226838 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:52.226796    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh" event={"ID":"b6351179-d647-424e-a7ba-87389b3df3fe","Type":"ContainerStarted","Data":"d2b7eda8126d239bf1c24a08f4fd8d5835c285ebfcc87174a692ee4ecc06c1ae"}
Sep 14 07:52:54.237273 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:54.237240    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh" event={"ID":"b6351179-d647-424e-a7ba-87389b3df3fe","Type":"ContainerStarted","Data":"16e4bb8471484608580cf953d5fc7acfda1ab2a0ec0718cd2ef2f539f5ce94ab"}
Sep 14 07:52:54.260092 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:54.260042    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vmkvh" podStartSLOduration=1.6954693010000002 podStartE2EDuration="3.26002437s" totalImagesPullingTime="1.564555069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:52:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:52:54.259304732 +0000 UTC m=+294.888192371" watchObservedRunningTime="2026-09-14 07:52:54.26002437 +0000 UTC m=+294.888911997"
Sep 14 07:52:57.130753 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.130723    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"]
Sep 14 07:52:57.151779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.151743    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"]
Sep 14 07:52:57.151945 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.151860    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"
Sep 14 07:52:57.154716 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.154687    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 07:52:57.155447 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.155394    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 07:52:57.155447 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.155394    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-q6vxq\""
Sep 14 07:52:57.337212 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.337175    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp8r8\" (UniqueName: \"kubernetes.io/projected/a9698b74-37b5-434b-9428-b8cfc92388e7-kube-api-access-dp8r8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p\" (UID: \"a9698b74-37b5-434b-9428-b8cfc92388e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"
Sep 14 07:52:57.337370 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.337228    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9698b74-37b5-434b-9428-b8cfc92388e7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p\" (UID: \"a9698b74-37b5-434b-9428-b8cfc92388e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"
Sep 14 07:52:57.337370 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.337334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9698b74-37b5-434b-9428-b8cfc92388e7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p\" (UID: \"a9698b74-37b5-434b-9428-b8cfc92388e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"
Sep 14 07:52:57.438603 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.438516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9698b74-37b5-434b-9428-b8cfc92388e7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p\" (UID: \"a9698b74-37b5-434b-9428-b8cfc92388e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"
Sep 14 07:52:57.438603 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.438579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp8r8\" (UniqueName: \"kubernetes.io/projected/a9698b74-37b5-434b-9428-b8cfc92388e7-kube-api-access-dp8r8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p\" (UID: \"a9698b74-37b5-434b-9428-b8cfc92388e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"
Sep 14 07:52:57.438792 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.438617    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9698b74-37b5-434b-9428-b8cfc92388e7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p\" (UID: \"a9698b74-37b5-434b-9428-b8cfc92388e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"
Sep 14 07:52:57.438932 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.438912    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9698b74-37b5-434b-9428-b8cfc92388e7-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p\" (UID: \"a9698b74-37b5-434b-9428-b8cfc92388e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"
Sep 14 07:52:57.438969 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.438950    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9698b74-37b5-434b-9428-b8cfc92388e7-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p\" (UID: \"a9698b74-37b5-434b-9428-b8cfc92388e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"
Sep 14 07:52:57.447048 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.447018    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp8r8\" (UniqueName: \"kubernetes.io/projected/a9698b74-37b5-434b-9428-b8cfc92388e7-kube-api-access-dp8r8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p\" (UID: \"a9698b74-37b5-434b-9428-b8cfc92388e7\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"
Sep 14 07:52:57.460948 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.460929    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"
Sep 14 07:52:57.749013 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:57.748989    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"]
Sep 14 07:52:57.750919 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:52:57.750892    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9698b74_37b5_434b_9428_b8cfc92388e7.slice/crio-2ecabfb4ef26a1966c983cff2ae5bb0b33aa479b7b68cbe72b06328a1d58078d WatchSource:0}: Error finding container 2ecabfb4ef26a1966c983cff2ae5bb0b33aa479b7b68cbe72b06328a1d58078d: Status 404 returned error can't find the container with id 2ecabfb4ef26a1966c983cff2ae5bb0b33aa479b7b68cbe72b06328a1d58078d
Sep 14 07:52:58.261829 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:58.261791    2563 generic.go:356] "Generic (PLEG): container finished" podID="a9698b74-37b5-434b-9428-b8cfc92388e7" containerID="5fa5cd1bfede7f233b9cbc37838a5256332c4a0da2127541bcf7c6aac0c37310" exitCode=0
Sep 14 07:52:58.262274 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:58.261865    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p" event={"ID":"a9698b74-37b5-434b-9428-b8cfc92388e7","Type":"ContainerDied","Data":"5fa5cd1bfede7f233b9cbc37838a5256332c4a0da2127541bcf7c6aac0c37310"}
Sep 14 07:52:58.262274 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:58.261939    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p" event={"ID":"a9698b74-37b5-434b-9428-b8cfc92388e7","Type":"ContainerStarted","Data":"2ecabfb4ef26a1966c983cff2ae5bb0b33aa479b7b68cbe72b06328a1d58078d"}
Sep 14 07:52:59.757234 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:59.757164    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 07:52:59.760924 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:59.760899    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 07:52:59.768518 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:52:59.768492    2563 kubelet.go:1648] "Image garbage collection succeeded"
Sep 14 07:53:00.271826 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:00.271789    2563 generic.go:356] "Generic (PLEG): container finished" podID="a9698b74-37b5-434b-9428-b8cfc92388e7" containerID="96151330c5ebcf6126d08c4b74e906083e228bcf97717374960d4e82d224f8b9" exitCode=0
Sep 14 07:53:00.272933 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:00.271868    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p" event={"ID":"a9698b74-37b5-434b-9428-b8cfc92388e7","Type":"ContainerDied","Data":"96151330c5ebcf6126d08c4b74e906083e228bcf97717374960d4e82d224f8b9"}
Sep 14 07:53:01.279270 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:01.279236    2563 generic.go:356] "Generic (PLEG): container finished" podID="a9698b74-37b5-434b-9428-b8cfc92388e7" containerID="b11fc0fdab0e388a6d62e298068b75d7fe65d6a0c4d3c39dd82e37ec32db533d" exitCode=0
Sep 14 07:53:01.279670 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:01.279314    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p" event={"ID":"a9698b74-37b5-434b-9428-b8cfc92388e7","Type":"ContainerDied","Data":"b11fc0fdab0e388a6d62e298068b75d7fe65d6a0c4d3c39dd82e37ec32db533d"}
Sep 14 07:53:02.412896 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:02.412868    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"
Sep 14 07:53:02.582856 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:02.582825    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp8r8\" (UniqueName: \"kubernetes.io/projected/a9698b74-37b5-434b-9428-b8cfc92388e7-kube-api-access-dp8r8\") pod \"a9698b74-37b5-434b-9428-b8cfc92388e7\" (UID: \"a9698b74-37b5-434b-9428-b8cfc92388e7\") "
Sep 14 07:53:02.583016 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:02.582870    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9698b74-37b5-434b-9428-b8cfc92388e7-util\") pod \"a9698b74-37b5-434b-9428-b8cfc92388e7\" (UID: \"a9698b74-37b5-434b-9428-b8cfc92388e7\") "
Sep 14 07:53:02.583016 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:02.582913    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9698b74-37b5-434b-9428-b8cfc92388e7-bundle\") pod \"a9698b74-37b5-434b-9428-b8cfc92388e7\" (UID: \"a9698b74-37b5-434b-9428-b8cfc92388e7\") "
Sep 14 07:53:02.583737 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:02.583710    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9698b74-37b5-434b-9428-b8cfc92388e7-bundle" (OuterVolumeSpecName: "bundle") pod "a9698b74-37b5-434b-9428-b8cfc92388e7" (UID: "a9698b74-37b5-434b-9428-b8cfc92388e7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:53:02.584990 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:02.584964    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9698b74-37b5-434b-9428-b8cfc92388e7-kube-api-access-dp8r8" (OuterVolumeSpecName: "kube-api-access-dp8r8") pod "a9698b74-37b5-434b-9428-b8cfc92388e7" (UID: "a9698b74-37b5-434b-9428-b8cfc92388e7"). InnerVolumeSpecName "kube-api-access-dp8r8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:53:02.588388 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:02.588368    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9698b74-37b5-434b-9428-b8cfc92388e7-util" (OuterVolumeSpecName: "util") pod "a9698b74-37b5-434b-9428-b8cfc92388e7" (UID: "a9698b74-37b5-434b-9428-b8cfc92388e7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:53:02.684064 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:02.684023    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp8r8\" (UniqueName: \"kubernetes.io/projected/a9698b74-37b5-434b-9428-b8cfc92388e7-kube-api-access-dp8r8\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:53:02.684064 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:02.684058    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9698b74-37b5-434b-9428-b8cfc92388e7-util\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:53:02.684064 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:02.684068    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9698b74-37b5-434b-9428-b8cfc92388e7-bundle\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:53:03.290854 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:03.290823    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p"
Sep 14 07:53:03.290854 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:03.290841    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdk28p" event={"ID":"a9698b74-37b5-434b-9428-b8cfc92388e7","Type":"ContainerDied","Data":"2ecabfb4ef26a1966c983cff2ae5bb0b33aa479b7b68cbe72b06328a1d58078d"}
Sep 14 07:53:03.291058 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:03.290890    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ecabfb4ef26a1966c983cff2ae5bb0b33aa479b7b68cbe72b06328a1d58078d"
Sep 14 07:53:11.540966 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.540929    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"]
Sep 14 07:53:11.541374 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.541288    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9698b74-37b5-434b-9428-b8cfc92388e7" containerName="util"
Sep 14 07:53:11.541374 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.541300    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9698b74-37b5-434b-9428-b8cfc92388e7" containerName="util"
Sep 14 07:53:11.541374 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.541314    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9698b74-37b5-434b-9428-b8cfc92388e7" containerName="extract"
Sep 14 07:53:11.541374 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.541324    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9698b74-37b5-434b-9428-b8cfc92388e7" containerName="extract"
Sep 14 07:53:11.541374 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.541334    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9698b74-37b5-434b-9428-b8cfc92388e7" containerName="pull"
Sep 14 07:53:11.541374 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.541340    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9698b74-37b5-434b-9428-b8cfc92388e7" containerName="pull"
Sep 14 07:53:11.541610 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.541447    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9698b74-37b5-434b-9428-b8cfc92388e7" containerName="extract"
Sep 14 07:53:11.550164 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.550138    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"
Sep 14 07:53:11.553087 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.552990    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-q6vxq\""
Sep 14 07:53:11.554169 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.554149    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 07:53:11.554320 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.554302    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 07:53:11.558183 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.558005    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"]
Sep 14 07:53:11.642259 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.642227    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/245e70fd-55a6-4d74-80ee-2b138ca51c19-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2\" (UID: \"245e70fd-55a6-4d74-80ee-2b138ca51c19\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"
Sep 14 07:53:11.642416 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.642279    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/245e70fd-55a6-4d74-80ee-2b138ca51c19-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2\" (UID: \"245e70fd-55a6-4d74-80ee-2b138ca51c19\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"
Sep 14 07:53:11.642416 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.642302    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-246qh\" (UniqueName: \"kubernetes.io/projected/245e70fd-55a6-4d74-80ee-2b138ca51c19-kube-api-access-246qh\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2\" (UID: \"245e70fd-55a6-4d74-80ee-2b138ca51c19\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"
Sep 14 07:53:11.742717 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.742676    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/245e70fd-55a6-4d74-80ee-2b138ca51c19-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2\" (UID: \"245e70fd-55a6-4d74-80ee-2b138ca51c19\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"
Sep 14 07:53:11.742871 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.742738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/245e70fd-55a6-4d74-80ee-2b138ca51c19-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2\" (UID: \"245e70fd-55a6-4d74-80ee-2b138ca51c19\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"
Sep 14 07:53:11.742871 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.742759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-246qh\" (UniqueName: \"kubernetes.io/projected/245e70fd-55a6-4d74-80ee-2b138ca51c19-kube-api-access-246qh\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2\" (UID: \"245e70fd-55a6-4d74-80ee-2b138ca51c19\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"
Sep 14 07:53:11.743092 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.743070    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/245e70fd-55a6-4d74-80ee-2b138ca51c19-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2\" (UID: \"245e70fd-55a6-4d74-80ee-2b138ca51c19\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"
Sep 14 07:53:11.743128 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.743080    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/245e70fd-55a6-4d74-80ee-2b138ca51c19-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2\" (UID: \"245e70fd-55a6-4d74-80ee-2b138ca51c19\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"
Sep 14 07:53:11.754489 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.754453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-246qh\" (UniqueName: \"kubernetes.io/projected/245e70fd-55a6-4d74-80ee-2b138ca51c19-kube-api-access-246qh\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2\" (UID: \"245e70fd-55a6-4d74-80ee-2b138ca51c19\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"
Sep 14 07:53:11.861095 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:11.861064    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"
Sep 14 07:53:12.130950 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:12.130925    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"]
Sep 14 07:53:12.133137 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:53:12.133110    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod245e70fd_55a6_4d74_80ee_2b138ca51c19.slice/crio-be5a2e70a89cc3c2928b297b684d97abd1d57d1eb5bd4b6bd962a8ee9421fc2f WatchSource:0}: Error finding container be5a2e70a89cc3c2928b297b684d97abd1d57d1eb5bd4b6bd962a8ee9421fc2f: Status 404 returned error can't find the container with id be5a2e70a89cc3c2928b297b684d97abd1d57d1eb5bd4b6bd962a8ee9421fc2f
Sep 14 07:53:12.333255 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:12.333224    2563 generic.go:356] "Generic (PLEG): container finished" podID="245e70fd-55a6-4d74-80ee-2b138ca51c19" containerID="e11ab176a982baaeb14db65f8028f930736b89c452354b04c9a2c59ca07c1e0a" exitCode=0
Sep 14 07:53:12.333428 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:12.333321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2" event={"ID":"245e70fd-55a6-4d74-80ee-2b138ca51c19","Type":"ContainerDied","Data":"e11ab176a982baaeb14db65f8028f930736b89c452354b04c9a2c59ca07c1e0a"}
Sep 14 07:53:12.333428 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:12.333373    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2" event={"ID":"245e70fd-55a6-4d74-80ee-2b138ca51c19","Type":"ContainerStarted","Data":"be5a2e70a89cc3c2928b297b684d97abd1d57d1eb5bd4b6bd962a8ee9421fc2f"}
Sep 14 07:53:12.333749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:12.333737    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:53:13.188423 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.188398    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"]
Sep 14 07:53:13.193110 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.193091    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.195546 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.195525    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 14 07:53:13.196303 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.196046    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 14 07:53:13.196303 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.196060    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 14 07:53:13.196303 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.196194    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 14 07:53:13.196569 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.196401    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-79xct\""
Sep 14 07:53:13.205042 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.205016    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"]
Sep 14 07:53:13.255315 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.255286    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.255485 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.255319    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxcfl\" (UniqueName: \"kubernetes.io/projected/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-kube-api-access-dxcfl\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.255485 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.255390    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-tmp\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.255485 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.255406    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.255485 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.255430    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-manifests\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.344249 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.344217    2563 generic.go:356] "Generic (PLEG): container finished" podID="245e70fd-55a6-4d74-80ee-2b138ca51c19" containerID="1532d7a36eec25610c3ea8b79decb89184e8e6f28c4aa142cff643bee2fd89ab" exitCode=0
Sep 14 07:53:13.344396 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.344297    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2" event={"ID":"245e70fd-55a6-4d74-80ee-2b138ca51c19","Type":"ContainerDied","Data":"1532d7a36eec25610c3ea8b79decb89184e8e6f28c4aa142cff643bee2fd89ab"}
Sep 14 07:53:13.356173 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.356141    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-tmp\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.356173 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.356172    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.356318 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.356203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-manifests\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.356409 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.356354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.356471 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.356428    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxcfl\" (UniqueName: \"kubernetes.io/projected/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-kube-api-access-dxcfl\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.356586 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.356556    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-manifests\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.358713 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.358635    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-tmp\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.358854 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.358834    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.359000 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.358982    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.367939 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.367917    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxcfl\" (UniqueName: \"kubernetes.io/projected/2a4a1d7c-fb45-43e7-a90e-3d300bee86a1-kube-api-access-dxcfl\") pod \"opendatahub-operator-controller-manager-84c4c5b687-nvmjc\" (UID: \"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.504935 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.504909    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:13.787072 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:13.787033    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"]
Sep 14 07:53:13.790870 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:53:13.790846    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a4a1d7c_fb45_43e7_a90e_3d300bee86a1.slice/crio-1062bde8bde3284b8dedb2a0582e45769a6f15f9b62922e6a5c261d0b83c7ba9 WatchSource:0}: Error finding container 1062bde8bde3284b8dedb2a0582e45769a6f15f9b62922e6a5c261d0b83c7ba9: Status 404 returned error can't find the container with id 1062bde8bde3284b8dedb2a0582e45769a6f15f9b62922e6a5c261d0b83c7ba9
Sep 14 07:53:14.356665 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:14.356024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc" event={"ID":"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1","Type":"ContainerStarted","Data":"1062bde8bde3284b8dedb2a0582e45769a6f15f9b62922e6a5c261d0b83c7ba9"}
Sep 14 07:53:14.363313 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:14.360865    2563 generic.go:356] "Generic (PLEG): container finished" podID="245e70fd-55a6-4d74-80ee-2b138ca51c19" containerID="81436aa4ba354b9e60cb221cbdda2b0bccd677ba8af5697ea78f74fac9c3f1fb" exitCode=0
Sep 14 07:53:14.363313 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:14.360931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2" event={"ID":"245e70fd-55a6-4d74-80ee-2b138ca51c19","Type":"ContainerDied","Data":"81436aa4ba354b9e60cb221cbdda2b0bccd677ba8af5697ea78f74fac9c3f1fb"}
Sep 14 07:53:16.691664 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:16.691623    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"
Sep 14 07:53:16.783207 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:16.783186    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/245e70fd-55a6-4d74-80ee-2b138ca51c19-util\") pod \"245e70fd-55a6-4d74-80ee-2b138ca51c19\" (UID: \"245e70fd-55a6-4d74-80ee-2b138ca51c19\") "
Sep 14 07:53:16.783312 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:16.783235    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-246qh\" (UniqueName: \"kubernetes.io/projected/245e70fd-55a6-4d74-80ee-2b138ca51c19-kube-api-access-246qh\") pod \"245e70fd-55a6-4d74-80ee-2b138ca51c19\" (UID: \"245e70fd-55a6-4d74-80ee-2b138ca51c19\") "
Sep 14 07:53:16.783312 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:16.783263    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/245e70fd-55a6-4d74-80ee-2b138ca51c19-bundle\") pod \"245e70fd-55a6-4d74-80ee-2b138ca51c19\" (UID: \"245e70fd-55a6-4d74-80ee-2b138ca51c19\") "
Sep 14 07:53:16.784220 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:16.784192    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/245e70fd-55a6-4d74-80ee-2b138ca51c19-bundle" (OuterVolumeSpecName: "bundle") pod "245e70fd-55a6-4d74-80ee-2b138ca51c19" (UID: "245e70fd-55a6-4d74-80ee-2b138ca51c19"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:53:16.785208 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:16.785183    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/245e70fd-55a6-4d74-80ee-2b138ca51c19-kube-api-access-246qh" (OuterVolumeSpecName: "kube-api-access-246qh") pod "245e70fd-55a6-4d74-80ee-2b138ca51c19" (UID: "245e70fd-55a6-4d74-80ee-2b138ca51c19"). InnerVolumeSpecName "kube-api-access-246qh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:53:16.789277 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:16.789242    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/245e70fd-55a6-4d74-80ee-2b138ca51c19-util" (OuterVolumeSpecName: "util") pod "245e70fd-55a6-4d74-80ee-2b138ca51c19" (UID: "245e70fd-55a6-4d74-80ee-2b138ca51c19"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:53:16.883865 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:16.883835    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/245e70fd-55a6-4d74-80ee-2b138ca51c19-util\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:53:16.883865 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:16.883859    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-246qh\" (UniqueName: \"kubernetes.io/projected/245e70fd-55a6-4d74-80ee-2b138ca51c19-kube-api-access-246qh\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:53:16.883865 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:16.883871    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/245e70fd-55a6-4d74-80ee-2b138ca51c19-bundle\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:53:17.378785 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:17.378754    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2"
Sep 14 07:53:17.378785 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:17.378770    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0qxfm2" event={"ID":"245e70fd-55a6-4d74-80ee-2b138ca51c19","Type":"ContainerDied","Data":"be5a2e70a89cc3c2928b297b684d97abd1d57d1eb5bd4b6bd962a8ee9421fc2f"}
Sep 14 07:53:17.379048 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:17.378815    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be5a2e70a89cc3c2928b297b684d97abd1d57d1eb5bd4b6bd962a8ee9421fc2f"
Sep 14 07:53:17.380414 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:17.380390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc" event={"ID":"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1","Type":"ContainerStarted","Data":"892828431ad6dc2d95afe1c718906d3894ba91fbf8cfece9c5ae5f786aa81432"}
Sep 14 07:53:18.386876 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:18.386841    2563 generic.go:356] "Generic (PLEG): container finished" podID="2a4a1d7c-fb45-43e7-a90e-3d300bee86a1" containerID="892828431ad6dc2d95afe1c718906d3894ba91fbf8cfece9c5ae5f786aa81432" exitCode=0
Sep 14 07:53:18.387260 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:18.386924    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc" event={"ID":"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1","Type":"ContainerDied","Data":"892828431ad6dc2d95afe1c718906d3894ba91fbf8cfece9c5ae5f786aa81432"}
Sep 14 07:53:19.393762 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:19.393728    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc" event={"ID":"2a4a1d7c-fb45-43e7-a90e-3d300bee86a1","Type":"ContainerStarted","Data":"ecec1b18446ee87b4acde84460f8a7b964d3faf3a678d166aae2f04d0916756c"}
Sep 14 07:53:19.394158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:19.393832    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:19.419048 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:19.418997    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc" podStartSLOduration=1.965211077 podStartE2EDuration="6.418979912s" totalImagesPullingTime="3.395133976s" totalInitContainerRuntime="1.058634859s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:19.417562713 +0000 UTC m=+320.046450360" watchObservedRunningTime="2026-09-14 07:53:19.418979912 +0000 UTC m=+320.047867538"
Sep 14 07:53:30.401088 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:30.401056    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-nvmjc"
Sep 14 07:53:32.842409 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.842372    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"]
Sep 14 07:53:32.842814 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.842698    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="245e70fd-55a6-4d74-80ee-2b138ca51c19" containerName="pull"
Sep 14 07:53:32.842814 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.842713    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="245e70fd-55a6-4d74-80ee-2b138ca51c19" containerName="pull"
Sep 14 07:53:32.842814 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.842725    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="245e70fd-55a6-4d74-80ee-2b138ca51c19" containerName="util"
Sep 14 07:53:32.842814 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.842731    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="245e70fd-55a6-4d74-80ee-2b138ca51c19" containerName="util"
Sep 14 07:53:32.842814 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.842744    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="245e70fd-55a6-4d74-80ee-2b138ca51c19" containerName="extract"
Sep 14 07:53:32.842814 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.842751    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="245e70fd-55a6-4d74-80ee-2b138ca51c19" containerName="extract"
Sep 14 07:53:32.842814 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.842805    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="245e70fd-55a6-4d74-80ee-2b138ca51c19" containerName="extract"
Sep 14 07:53:32.847006 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.846983    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"
Sep 14 07:53:32.849239 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.849221    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 14 07:53:32.849364 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.849344    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-nplqb\""
Sep 14 07:53:32.849710 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.849695    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 14 07:53:32.858269 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.858241    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"]
Sep 14 07:53:32.909420 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.909388    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7jk2\" (UniqueName: \"kubernetes.io/projected/64cd7e59-9eb7-48f2-bbcd-d12df7334aca-kube-api-access-h7jk2\") pod \"kube-auth-proxy-86f4846dcd-tnz86\" (UID: \"64cd7e59-9eb7-48f2-bbcd-d12df7334aca\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"
Sep 14 07:53:32.909594 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.909484    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64cd7e59-9eb7-48f2-bbcd-d12df7334aca-tmp\") pod \"kube-auth-proxy-86f4846dcd-tnz86\" (UID: \"64cd7e59-9eb7-48f2-bbcd-d12df7334aca\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"
Sep 14 07:53:32.909594 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:32.909521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64cd7e59-9eb7-48f2-bbcd-d12df7334aca-tls-certs\") pod \"kube-auth-proxy-86f4846dcd-tnz86\" (UID: \"64cd7e59-9eb7-48f2-bbcd-d12df7334aca\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"
Sep 14 07:53:33.010372 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:33.010336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7jk2\" (UniqueName: \"kubernetes.io/projected/64cd7e59-9eb7-48f2-bbcd-d12df7334aca-kube-api-access-h7jk2\") pod \"kube-auth-proxy-86f4846dcd-tnz86\" (UID: \"64cd7e59-9eb7-48f2-bbcd-d12df7334aca\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"
Sep 14 07:53:33.010550 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:33.010405    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64cd7e59-9eb7-48f2-bbcd-d12df7334aca-tmp\") pod \"kube-auth-proxy-86f4846dcd-tnz86\" (UID: \"64cd7e59-9eb7-48f2-bbcd-d12df7334aca\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"
Sep 14 07:53:33.010550 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:33.010429    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64cd7e59-9eb7-48f2-bbcd-d12df7334aca-tls-certs\") pod \"kube-auth-proxy-86f4846dcd-tnz86\" (UID: \"64cd7e59-9eb7-48f2-bbcd-d12df7334aca\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"
Sep 14 07:53:33.012727 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:33.012698    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64cd7e59-9eb7-48f2-bbcd-d12df7334aca-tmp\") pod \"kube-auth-proxy-86f4846dcd-tnz86\" (UID: \"64cd7e59-9eb7-48f2-bbcd-d12df7334aca\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"
Sep 14 07:53:33.012927 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:33.012910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64cd7e59-9eb7-48f2-bbcd-d12df7334aca-tls-certs\") pod \"kube-auth-proxy-86f4846dcd-tnz86\" (UID: \"64cd7e59-9eb7-48f2-bbcd-d12df7334aca\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"
Sep 14 07:53:33.019822 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:33.019801    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7jk2\" (UniqueName: \"kubernetes.io/projected/64cd7e59-9eb7-48f2-bbcd-d12df7334aca-kube-api-access-h7jk2\") pod \"kube-auth-proxy-86f4846dcd-tnz86\" (UID: \"64cd7e59-9eb7-48f2-bbcd-d12df7334aca\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"
Sep 14 07:53:33.156816 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:33.156721    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"
Sep 14 07:53:33.441272 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:33.441246    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86"]
Sep 14 07:53:33.443289 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:53:33.443265    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64cd7e59_9eb7_48f2_bbcd_d12df7334aca.slice/crio-b455457bf0e6e3e86f6e32ba9cc5467a769312a0b73727755b06ab1bbdd7efca WatchSource:0}: Error finding container b455457bf0e6e3e86f6e32ba9cc5467a769312a0b73727755b06ab1bbdd7efca: Status 404 returned error can't find the container with id b455457bf0e6e3e86f6e32ba9cc5467a769312a0b73727755b06ab1bbdd7efca
Sep 14 07:53:33.459359 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:33.459332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86" event={"ID":"64cd7e59-9eb7-48f2-bbcd-d12df7334aca","Type":"ContainerStarted","Data":"b455457bf0e6e3e86f6e32ba9cc5467a769312a0b73727755b06ab1bbdd7efca"}
Sep 14 07:53:34.187658 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.187612    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"]
Sep 14 07:53:34.191128 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.191108    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.193869 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.193825    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 14 07:53:34.194277 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.194177    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 07:53:34.194277 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.194186    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 14 07:53:34.194418 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.194330    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jw96t\""
Sep 14 07:53:34.194494 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.194476    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 14 07:53:34.205545 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.205523    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"]
Sep 14 07:53:34.221419 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.221218    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk8wc\" (UniqueName: \"kubernetes.io/projected/d91f171e-e1d4-4748-8471-a6d75f6d9297-kube-api-access-jk8wc\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.221419 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.221262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.221419 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.221317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d91f171e-e1d4-4748-8471-a6d75f6d9297-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.221419 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.221342    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.221419 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.221385    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.221419 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.221414    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.221861 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.221452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.322052 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.322004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d91f171e-e1d4-4748-8471-a6d75f6d9297-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.322052 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.322054    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.322306 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.322093    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.322306 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.322121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.322306 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.322171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.322306 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.322242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk8wc\" (UniqueName: \"kubernetes.io/projected/d91f171e-e1d4-4748-8471-a6d75f6d9297-kube-api-access-jk8wc\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.322306 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.322269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.323158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.323098    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.325085 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.325061    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.325232 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.325207    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.325437 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.325413    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d91f171e-e1d4-4748-8471-a6d75f6d9297-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.326054 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.326015    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.342760 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.342734    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk8wc\" (UniqueName: \"kubernetes.io/projected/d91f171e-e1d4-4748-8471-a6d75f6d9297-kube-api-access-jk8wc\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.343189 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.343167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-txkrm\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.503719 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.503604    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:34.793907 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:34.793826    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"]
Sep 14 07:53:34.795746 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:53:34.795715    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd91f171e_e1d4_4748_8471_a6d75f6d9297.slice/crio-54e9d120ba616ddaeba12dcb3b47b59d49a846ec9f24a09e45eba9f1daeeb3e4 WatchSource:0}: Error finding container 54e9d120ba616ddaeba12dcb3b47b59d49a846ec9f24a09e45eba9f1daeeb3e4: Status 404 returned error can't find the container with id 54e9d120ba616ddaeba12dcb3b47b59d49a846ec9f24a09e45eba9f1daeeb3e4
Sep 14 07:53:35.477172 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:35.477130    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm" event={"ID":"d91f171e-e1d4-4748-8471-a6d75f6d9297","Type":"ContainerStarted","Data":"54e9d120ba616ddaeba12dcb3b47b59d49a846ec9f24a09e45eba9f1daeeb3e4"}
Sep 14 07:53:36.485687 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:36.485623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86" event={"ID":"64cd7e59-9eb7-48f2-bbcd-d12df7334aca","Type":"ContainerStarted","Data":"2b0151ea80d34081b1740e48ce1a6a69763b2ba78a2ea561747d4932e33980b4"}
Sep 14 07:53:36.512579 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:36.512521    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-86f4846dcd-tnz86" podStartSLOduration=1.659879703 podStartE2EDuration="4.51249909s" totalImagesPullingTime="2.852619387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:36.508414 +0000 UTC m=+337.137301628" watchObservedRunningTime="2026-09-14 07:53:36.51249909 +0000 UTC m=+337.141386718"
Sep 14 07:53:38.091687 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:38.091621    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 07:53:38.091977 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:38.091714    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 07:53:38.498548 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:38.498453    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm" event={"ID":"d91f171e-e1d4-4748-8471-a6d75f6d9297","Type":"ContainerStarted","Data":"3e9062ee0ae65b2e45e81caa9d2659edabf576703afb9b6148eab65ec20bb0a0"}
Sep 14 07:53:38.498715 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:38.498560    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:38.520834 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:38.520782    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm" podStartSLOduration=1.227608067 podStartE2EDuration="4.520765214s" totalImagesPullingTime="3.293157147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:38.519925305 +0000 UTC m=+339.148812950" watchObservedRunningTime="2026-09-14 07:53:38.520765214 +0000 UTC m=+339.149652843"
Sep 14 07:53:39.504399 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:39.504368    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:50.706204 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.706166    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"]
Sep 14 07:53:50.713022 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.712997    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.725587 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.725557    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"]
Sep 14 07:53:50.765156 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.765128    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/caea3af0-2fe0-4035-8098-21118882dff1-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.765310 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.765160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8frv8\" (UniqueName: \"kubernetes.io/projected/caea3af0-2fe0-4035-8098-21118882dff1-kube-api-access-8frv8\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.765310 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.765212    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/caea3af0-2fe0-4035-8098-21118882dff1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.765310 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.765242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/caea3af0-2fe0-4035-8098-21118882dff1-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.765310 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.765265    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caea3af0-2fe0-4035-8098-21118882dff1-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.765310 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.765298    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/caea3af0-2fe0-4035-8098-21118882dff1-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.765485 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.765357    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/caea3af0-2fe0-4035-8098-21118882dff1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.866057 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.866027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/caea3af0-2fe0-4035-8098-21118882dff1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.866246 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.866072    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/caea3af0-2fe0-4035-8098-21118882dff1-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.866246 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.866092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8frv8\" (UniqueName: \"kubernetes.io/projected/caea3af0-2fe0-4035-8098-21118882dff1-kube-api-access-8frv8\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.866246 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.866131    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/caea3af0-2fe0-4035-8098-21118882dff1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.866246 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.866155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/caea3af0-2fe0-4035-8098-21118882dff1-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.866246 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.866182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caea3af0-2fe0-4035-8098-21118882dff1-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.866246 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.866198    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/caea3af0-2fe0-4035-8098-21118882dff1-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.866977 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.866946    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/caea3af0-2fe0-4035-8098-21118882dff1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.868539 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.868511    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/caea3af0-2fe0-4035-8098-21118882dff1-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.868680 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.868546    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/caea3af0-2fe0-4035-8098-21118882dff1-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.868680 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.868544    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/caea3af0-2fe0-4035-8098-21118882dff1-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.868680 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.868595    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/caea3af0-2fe0-4035-8098-21118882dff1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.875329 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.875304    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caea3af0-2fe0-4035-8098-21118882dff1-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:50.875429 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:50.875405    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8frv8\" (UniqueName: \"kubernetes.io/projected/caea3af0-2fe0-4035-8098-21118882dff1-kube-api-access-8frv8\") pod \"istiod-openshift-gateway-5677ccb49b-r2gfm\" (UID: \"caea3af0-2fe0-4035-8098-21118882dff1\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:51.022288 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:51.022189    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:51.294539 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:51.294511    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"]
Sep 14 07:53:51.296189 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:53:51.296162    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaea3af0_2fe0_4035_8098_21118882dff1.slice/crio-5039977a366c57e05206c57f82f5e8a106294f41c423dba0e683985e3c290dcd WatchSource:0}: Error finding container 5039977a366c57e05206c57f82f5e8a106294f41c423dba0e683985e3c290dcd: Status 404 returned error can't find the container with id 5039977a366c57e05206c57f82f5e8a106294f41c423dba0e683985e3c290dcd
Sep 14 07:53:51.299353 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:51.299318    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 07:53:51.299417 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:51.299392    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 07:53:51.558714 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:51.558609    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm" event={"ID":"caea3af0-2fe0-4035-8098-21118882dff1","Type":"ContainerStarted","Data":"6c4494cc2a3cbc729a119ef8ff0a7f985424b9a2d27e1e3c7bb5480ad4e6900a"}
Sep 14 07:53:51.558714 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:51.558691    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:51.558714 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:51.558713    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm" event={"ID":"caea3af0-2fe0-4035-8098-21118882dff1","Type":"ContainerStarted","Data":"5039977a366c57e05206c57f82f5e8a106294f41c423dba0e683985e3c290dcd"}
Sep 14 07:53:51.588962 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:51.588898    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm" podStartSLOduration=1.588876493 podStartE2EDuration="1.588876493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:51.587478643 +0000 UTC m=+352.216366302" watchObservedRunningTime="2026-09-14 07:53:51.588876493 +0000 UTC m=+352.217764120"
Sep 14 07:53:52.565170 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.565097    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-r2gfm"
Sep 14 07:53:52.658447 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.658413    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"]
Sep 14 07:53:52.658664 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.658629    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm" podUID="d91f171e-e1d4-4748-8471-a6d75f6d9297" containerName="discovery" containerID="cri-o://3e9062ee0ae65b2e45e81caa9d2659edabf576703afb9b6148eab65ec20bb0a0" gracePeriod=30
Sep 14 07:53:52.901801 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.901766    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:52.987983 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.987950    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d91f171e-e1d4-4748-8471-a6d75f6d9297-local-certs\") pod \"d91f171e-e1d4-4748-8471-a6d75f6d9297\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") "
Sep 14 07:53:52.988168 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.987992    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jk8wc\" (UniqueName: \"kubernetes.io/projected/d91f171e-e1d4-4748-8471-a6d75f6d9297-kube-api-access-jk8wc\") pod \"d91f171e-e1d4-4748-8471-a6d75f6d9297\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") "
Sep 14 07:53:52.988168 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.988041    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-csr-dns-cert\") pod \"d91f171e-e1d4-4748-8471-a6d75f6d9297\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") "
Sep 14 07:53:52.988168 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.988061    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-cacerts\") pod \"d91f171e-e1d4-4748-8471-a6d75f6d9297\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") "
Sep 14 07:53:52.988168 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.988104    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-token\") pod \"d91f171e-e1d4-4748-8471-a6d75f6d9297\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") "
Sep 14 07:53:52.988168 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.988154    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-csr-ca-configmap\") pod \"d91f171e-e1d4-4748-8471-a6d75f6d9297\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") "
Sep 14 07:53:52.988433 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.988184    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-kubeconfig\") pod \"d91f171e-e1d4-4748-8471-a6d75f6d9297\" (UID: \"d91f171e-e1d4-4748-8471-a6d75f6d9297\") "
Sep 14 07:53:52.988903 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.988869    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "d91f171e-e1d4-4748-8471-a6d75f6d9297" (UID: "d91f171e-e1d4-4748-8471-a6d75f6d9297"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:53:52.990506 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.990481    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d91f171e-e1d4-4748-8471-a6d75f6d9297-local-certs" (OuterVolumeSpecName: "local-certs") pod "d91f171e-e1d4-4748-8471-a6d75f6d9297" (UID: "d91f171e-e1d4-4748-8471-a6d75f6d9297"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:53:52.990677 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.990636    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-cacerts" (OuterVolumeSpecName: "cacerts") pod "d91f171e-e1d4-4748-8471-a6d75f6d9297" (UID: "d91f171e-e1d4-4748-8471-a6d75f6d9297"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:53:52.990785 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.990752    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "d91f171e-e1d4-4748-8471-a6d75f6d9297" (UID: "d91f171e-e1d4-4748-8471-a6d75f6d9297"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:53:52.990897 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.990785    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "d91f171e-e1d4-4748-8471-a6d75f6d9297" (UID: "d91f171e-e1d4-4748-8471-a6d75f6d9297"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:53:52.991050 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.991024    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d91f171e-e1d4-4748-8471-a6d75f6d9297-kube-api-access-jk8wc" (OuterVolumeSpecName: "kube-api-access-jk8wc") pod "d91f171e-e1d4-4748-8471-a6d75f6d9297" (UID: "d91f171e-e1d4-4748-8471-a6d75f6d9297"). InnerVolumeSpecName "kube-api-access-jk8wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:53:52.991126 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:52.991098    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-token" (OuterVolumeSpecName: "istio-token") pod "d91f171e-e1d4-4748-8471-a6d75f6d9297" (UID: "d91f171e-e1d4-4748-8471-a6d75f6d9297"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:53:53.089079 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.089054    2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d91f171e-e1d4-4748-8471-a6d75f6d9297-local-certs\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:53:53.089079 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.089079    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk8wc\" (UniqueName: \"kubernetes.io/projected/d91f171e-e1d4-4748-8471-a6d75f6d9297-kube-api-access-jk8wc\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:53:53.089227 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.089089    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-csr-dns-cert\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:53:53.089227 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.089098    2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-cacerts\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:53:53.089227 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.089108    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-token\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:53:53.089227 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.089117    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-csr-ca-configmap\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:53:53.089227 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.089124    2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d91f171e-e1d4-4748-8471-a6d75f6d9297-istio-kubeconfig\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:53:53.570131 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.570032    2563 generic.go:356] "Generic (PLEG): container finished" podID="d91f171e-e1d4-4748-8471-a6d75f6d9297" containerID="3e9062ee0ae65b2e45e81caa9d2659edabf576703afb9b6148eab65ec20bb0a0" exitCode=0
Sep 14 07:53:53.570131 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.570084    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"
Sep 14 07:53:53.570678 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.570131    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm" event={"ID":"d91f171e-e1d4-4748-8471-a6d75f6d9297","Type":"ContainerDied","Data":"3e9062ee0ae65b2e45e81caa9d2659edabf576703afb9b6148eab65ec20bb0a0"}
Sep 14 07:53:53.570678 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.570228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm" event={"ID":"d91f171e-e1d4-4748-8471-a6d75f6d9297","Type":"ContainerDied","Data":"54e9d120ba616ddaeba12dcb3b47b59d49a846ec9f24a09e45eba9f1daeeb3e4"}
Sep 14 07:53:53.570678 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.570264    2563 scope.go:117] "RemoveContainer" containerID="3e9062ee0ae65b2e45e81caa9d2659edabf576703afb9b6148eab65ec20bb0a0"
Sep 14 07:53:53.579792 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.579775    2563 scope.go:117] "RemoveContainer" containerID="3e9062ee0ae65b2e45e81caa9d2659edabf576703afb9b6148eab65ec20bb0a0"
Sep 14 07:53:53.580086 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:53:53.580067    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e9062ee0ae65b2e45e81caa9d2659edabf576703afb9b6148eab65ec20bb0a0\": container with ID starting with 3e9062ee0ae65b2e45e81caa9d2659edabf576703afb9b6148eab65ec20bb0a0 not found: ID does not exist" containerID="3e9062ee0ae65b2e45e81caa9d2659edabf576703afb9b6148eab65ec20bb0a0"
Sep 14 07:53:53.580154 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.580094    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e9062ee0ae65b2e45e81caa9d2659edabf576703afb9b6148eab65ec20bb0a0"} err="failed to get container status \"3e9062ee0ae65b2e45e81caa9d2659edabf576703afb9b6148eab65ec20bb0a0\": rpc error: code = NotFound desc = could not find container \"3e9062ee0ae65b2e45e81caa9d2659edabf576703afb9b6148eab65ec20bb0a0\": container with ID starting with 3e9062ee0ae65b2e45e81caa9d2659edabf576703afb9b6148eab65ec20bb0a0 not found: ID does not exist"
Sep 14 07:53:53.597869 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.597844    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"]
Sep 14 07:53:53.606547 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.606519    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-txkrm"]
Sep 14 07:53:53.835536 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.835508    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh"]
Sep 14 07:53:53.835977 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.835951    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d91f171e-e1d4-4748-8471-a6d75f6d9297" containerName="discovery"
Sep 14 07:53:53.835977 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.835975    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d91f171e-e1d4-4748-8471-a6d75f6d9297" containerName="discovery"
Sep 14 07:53:53.836124 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.836052    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d91f171e-e1d4-4748-8471-a6d75f6d9297" containerName="discovery"
Sep 14 07:53:53.840721 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.840702    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh"
Sep 14 07:53:53.842940 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.842922    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-qjnhk\""
Sep 14 07:53:53.843171 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.843156    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 14 07:53:53.851290 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.851265    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh"]
Sep 14 07:53:53.885513 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.885480    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d91f171e-e1d4-4748-8471-a6d75f6d9297" path="/var/lib/kubelet/pods/d91f171e-e1d4-4748-8471-a6d75f6d9297/volumes"
Sep 14 07:53:53.996221 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.996188    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3ddd337-89aa-407a-ac0a-8d503cc3ef96-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6f9xh\" (UID: \"a3ddd337-89aa-407a-ac0a-8d503cc3ef96\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh"
Sep 14 07:53:53.996369 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:53.996293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhrtv\" (UniqueName: \"kubernetes.io/projected/a3ddd337-89aa-407a-ac0a-8d503cc3ef96-kube-api-access-zhrtv\") pod \"llmisvc-controller-manager-777c6bd7d5-6f9xh\" (UID: \"a3ddd337-89aa-407a-ac0a-8d503cc3ef96\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh"
Sep 14 07:53:54.097448 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:54.097352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhrtv\" (UniqueName: \"kubernetes.io/projected/a3ddd337-89aa-407a-ac0a-8d503cc3ef96-kube-api-access-zhrtv\") pod \"llmisvc-controller-manager-777c6bd7d5-6f9xh\" (UID: \"a3ddd337-89aa-407a-ac0a-8d503cc3ef96\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh"
Sep 14 07:53:54.097448 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:54.097422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3ddd337-89aa-407a-ac0a-8d503cc3ef96-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6f9xh\" (UID: \"a3ddd337-89aa-407a-ac0a-8d503cc3ef96\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh"
Sep 14 07:53:54.099896 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:54.099872    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3ddd337-89aa-407a-ac0a-8d503cc3ef96-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6f9xh\" (UID: \"a3ddd337-89aa-407a-ac0a-8d503cc3ef96\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh"
Sep 14 07:53:54.111092 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:54.111066    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhrtv\" (UniqueName: \"kubernetes.io/projected/a3ddd337-89aa-407a-ac0a-8d503cc3ef96-kube-api-access-zhrtv\") pod \"llmisvc-controller-manager-777c6bd7d5-6f9xh\" (UID: \"a3ddd337-89aa-407a-ac0a-8d503cc3ef96\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh"
Sep 14 07:53:54.152179 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:54.152151    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh"
Sep 14 07:53:54.439146 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:54.439114    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh"]
Sep 14 07:53:54.440056 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:53:54.440030    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda3ddd337_89aa_407a_ac0a_8d503cc3ef96.slice/crio-91a1830c9e763d042553d2c59b41235e17c44e633f77bd8e5cfd29b40d7143bd WatchSource:0}: Error finding container 91a1830c9e763d042553d2c59b41235e17c44e633f77bd8e5cfd29b40d7143bd: Status 404 returned error can't find the container with id 91a1830c9e763d042553d2c59b41235e17c44e633f77bd8e5cfd29b40d7143bd
Sep 14 07:53:54.579114 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:54.579082    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh" event={"ID":"a3ddd337-89aa-407a-ac0a-8d503cc3ef96","Type":"ContainerStarted","Data":"91a1830c9e763d042553d2c59b41235e17c44e633f77bd8e5cfd29b40d7143bd"}
Sep 14 07:53:57.596891 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:57.596852    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh" event={"ID":"a3ddd337-89aa-407a-ac0a-8d503cc3ef96","Type":"ContainerStarted","Data":"b3ab289192edfb7f79eaff3c6eef320e251d1520f3aeaf8df00b6c153ed17e25"}
Sep 14 07:53:57.597265 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:53:57.596974    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh"
Sep 14 07:54:28.605379 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:28.605339    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh"
Sep 14 07:54:28.624911 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:28.624849    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6f9xh" podStartSLOduration=32.890372364 podStartE2EDuration="35.624829228s" totalImagesPullingTime="2.734456864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:57.619073579 +0000 UTC m=+358.247961229" watchObservedRunningTime="2026-09-14 07:54:28.624829228 +0000 UTC m=+389.253716856"
Sep 14 07:54:44.162904 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.162869    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7679fc7b7c-hhqbr"]
Sep 14 07:54:44.173045 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.173021    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.174772 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.174744    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7679fc7b7c-hhqbr"]
Sep 14 07:54:44.175330 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.175312    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 14 07:54:44.175418 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.175355    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 14 07:54:44.175971 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.175817    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dfmfg\""
Sep 14 07:54:44.175971 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.175848    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 14 07:54:44.175971 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.175901    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 14 07:54:44.175971 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.175902    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 14 07:54:44.176220 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.176035    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 14 07:54:44.176220 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.176131    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 14 07:54:44.180851 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.180832    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 14 07:54:44.304466 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.304428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-console-oauth-config\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.304466 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.304468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-console-serving-cert\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.304705 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.304532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-oauth-serving-cert\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.304705 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.304610    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-trusted-ca-bundle\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.304705 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.304632    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-console-config\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.304705 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.304674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-service-ca\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.304705 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.304693    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7b4k\" (UniqueName: \"kubernetes.io/projected/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-kube-api-access-p7b4k\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.405500 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.405465    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-service-ca\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.405500 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.405501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7b4k\" (UniqueName: \"kubernetes.io/projected/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-kube-api-access-p7b4k\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.405767 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.405537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-console-oauth-config\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.405767 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.405626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-console-serving-cert\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.405767 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.405666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-oauth-serving-cert\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.405767 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.405705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-trusted-ca-bundle\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.405767 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.405722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-console-config\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.406370 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.406341    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-service-ca\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.406502 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.406391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-oauth-serving-cert\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.406502 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.406391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-console-config\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.406620 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.406560    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-trusted-ca-bundle\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.408126 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.408102    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-console-serving-cert\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.408260 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.408240    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-console-oauth-config\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.413979 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.413937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7b4k\" (UniqueName: \"kubernetes.io/projected/b959c0f7-7a0d-434a-805a-9bc9e8075fd3-kube-api-access-p7b4k\") pod \"console-7679fc7b7c-hhqbr\" (UID: \"b959c0f7-7a0d-434a-805a-9bc9e8075fd3\") " pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.484525 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.484494    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:44.767053 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.767029    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7679fc7b7c-hhqbr"]
Sep 14 07:54:44.768935 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:54:44.768901    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb959c0f7_7a0d_434a_805a_9bc9e8075fd3.slice/crio-b4be3b5b21aef138f211f24eca9b59042d5594b57394eb1dd1be103b965f7fa8 WatchSource:0}: Error finding container b4be3b5b21aef138f211f24eca9b59042d5594b57394eb1dd1be103b965f7fa8: Status 404 returned error can't find the container with id b4be3b5b21aef138f211f24eca9b59042d5594b57394eb1dd1be103b965f7fa8
Sep 14 07:54:44.818701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:44.818673    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7679fc7b7c-hhqbr" event={"ID":"b959c0f7-7a0d-434a-805a-9bc9e8075fd3","Type":"ContainerStarted","Data":"b4be3b5b21aef138f211f24eca9b59042d5594b57394eb1dd1be103b965f7fa8"}
Sep 14 07:54:45.824971 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:45.824931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7679fc7b7c-hhqbr" event={"ID":"b959c0f7-7a0d-434a-805a-9bc9e8075fd3","Type":"ContainerStarted","Data":"d44fed6fc19b3c203c0ce3a4651a2a0722b5163f77afde87ef51ee89429062ba"}
Sep 14 07:54:45.844898 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:45.844850    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7679fc7b7c-hhqbr" podStartSLOduration=1.844833942 podStartE2EDuration="1.844833942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:54:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:54:45.843712729 +0000 UTC m=+406.472600380" watchObservedRunningTime="2026-09-14 07:54:45.844833942 +0000 UTC m=+406.473721569"
Sep 14 07:54:47.828992 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:47.828959    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"]
Sep 14 07:54:47.832568 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:47.832539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"
Sep 14 07:54:47.834670 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:47.834630    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 14 07:54:47.834800 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:47.834691    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-q6vxq\""
Sep 14 07:54:47.835203 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:47.835179    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 14 07:54:47.839945 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:47.839921    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"]
Sep 14 07:54:47.936324 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:47.936293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2cbc335-cad7-4265-8522-6c7e2021ca32-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z\" (UID: \"f2cbc335-cad7-4265-8522-6c7e2021ca32\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"
Sep 14 07:54:47.936463 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:47.936395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krmh8\" (UniqueName: \"kubernetes.io/projected/f2cbc335-cad7-4265-8522-6c7e2021ca32-kube-api-access-krmh8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z\" (UID: \"f2cbc335-cad7-4265-8522-6c7e2021ca32\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"
Sep 14 07:54:47.936463 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:47.936427    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2cbc335-cad7-4265-8522-6c7e2021ca32-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z\" (UID: \"f2cbc335-cad7-4265-8522-6c7e2021ca32\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"
Sep 14 07:54:47.983390 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:47.983358    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"]
Sep 14 07:54:47.986829 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:47.986811    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"
Sep 14 07:54:48.002627 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.002605    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"]
Sep 14 07:54:48.037692 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.037663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84e3b904-d8df-43fa-931d-3e53fff06c41-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4\" (UID: \"84e3b904-d8df-43fa-931d-3e53fff06c41\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"
Sep 14 07:54:48.037823 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.037721    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krmh8\" (UniqueName: \"kubernetes.io/projected/f2cbc335-cad7-4265-8522-6c7e2021ca32-kube-api-access-krmh8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z\" (UID: \"f2cbc335-cad7-4265-8522-6c7e2021ca32\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"
Sep 14 07:54:48.037823 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.037744    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2cbc335-cad7-4265-8522-6c7e2021ca32-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z\" (UID: \"f2cbc335-cad7-4265-8522-6c7e2021ca32\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"
Sep 14 07:54:48.037823 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.037760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxt4g\" (UniqueName: \"kubernetes.io/projected/84e3b904-d8df-43fa-931d-3e53fff06c41-kube-api-access-qxt4g\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4\" (UID: \"84e3b904-d8df-43fa-931d-3e53fff06c41\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"
Sep 14 07:54:48.037823 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.037788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84e3b904-d8df-43fa-931d-3e53fff06c41-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4\" (UID: \"84e3b904-d8df-43fa-931d-3e53fff06c41\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"
Sep 14 07:54:48.037823 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.037811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2cbc335-cad7-4265-8522-6c7e2021ca32-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z\" (UID: \"f2cbc335-cad7-4265-8522-6c7e2021ca32\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"
Sep 14 07:54:48.038096 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.038073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2cbc335-cad7-4265-8522-6c7e2021ca32-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z\" (UID: \"f2cbc335-cad7-4265-8522-6c7e2021ca32\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"
Sep 14 07:54:48.038144 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.038128    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2cbc335-cad7-4265-8522-6c7e2021ca32-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z\" (UID: \"f2cbc335-cad7-4265-8522-6c7e2021ca32\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"
Sep 14 07:54:48.048305 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.048277    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krmh8\" (UniqueName: \"kubernetes.io/projected/f2cbc335-cad7-4265-8522-6c7e2021ca32-kube-api-access-krmh8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z\" (UID: \"f2cbc335-cad7-4265-8522-6c7e2021ca32\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"
Sep 14 07:54:48.055449 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.055429    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"]
Sep 14 07:54:48.059047 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.059033    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"
Sep 14 07:54:48.066718 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.066665    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"]
Sep 14 07:54:48.123493 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.123420    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"]
Sep 14 07:54:48.127011 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.126994    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"
Sep 14 07:54:48.136103 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.136078    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"]
Sep 14 07:54:48.138967 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.138944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84e3b904-d8df-43fa-931d-3e53fff06c41-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4\" (UID: \"84e3b904-d8df-43fa-931d-3e53fff06c41\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"
Sep 14 07:54:48.139080 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.138982    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj575\" (UniqueName: \"kubernetes.io/projected/c7f70cb1-b99f-401e-a977-6feccacf90ab-kube-api-access-mj575\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75\" (UID: \"c7f70cb1-b99f-401e-a977-6feccacf90ab\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"
Sep 14 07:54:48.139080 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.139050    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxt4g\" (UniqueName: \"kubernetes.io/projected/84e3b904-d8df-43fa-931d-3e53fff06c41-kube-api-access-qxt4g\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4\" (UID: \"84e3b904-d8df-43fa-931d-3e53fff06c41\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"
Sep 14 07:54:48.139196 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.139089    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f70cb1-b99f-401e-a977-6feccacf90ab-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75\" (UID: \"c7f70cb1-b99f-401e-a977-6feccacf90ab\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"
Sep 14 07:54:48.139246 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.139208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84e3b904-d8df-43fa-931d-3e53fff06c41-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4\" (UID: \"84e3b904-d8df-43fa-931d-3e53fff06c41\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"
Sep 14 07:54:48.139311 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.139285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f70cb1-b99f-401e-a977-6feccacf90ab-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75\" (UID: \"c7f70cb1-b99f-401e-a977-6feccacf90ab\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"
Sep 14 07:54:48.139393 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.139292    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84e3b904-d8df-43fa-931d-3e53fff06c41-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4\" (UID: \"84e3b904-d8df-43fa-931d-3e53fff06c41\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"
Sep 14 07:54:48.139513 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.139460    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84e3b904-d8df-43fa-931d-3e53fff06c41-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4\" (UID: \"84e3b904-d8df-43fa-931d-3e53fff06c41\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"
Sep 14 07:54:48.143906 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.143885    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"
Sep 14 07:54:48.151095 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.151074    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxt4g\" (UniqueName: \"kubernetes.io/projected/84e3b904-d8df-43fa-931d-3e53fff06c41-kube-api-access-qxt4g\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4\" (UID: \"84e3b904-d8df-43fa-931d-3e53fff06c41\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"
Sep 14 07:54:48.240454 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.240422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f70cb1-b99f-401e-a977-6feccacf90ab-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75\" (UID: \"c7f70cb1-b99f-401e-a977-6feccacf90ab\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"
Sep 14 07:54:48.240636 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.240474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx\" (UID: \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"
Sep 14 07:54:48.240636 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.240524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj575\" (UniqueName: \"kubernetes.io/projected/c7f70cb1-b99f-401e-a977-6feccacf90ab-kube-api-access-mj575\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75\" (UID: \"c7f70cb1-b99f-401e-a977-6feccacf90ab\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"
Sep 14 07:54:48.240636 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.240570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vjlb\" (UniqueName: \"kubernetes.io/projected/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-kube-api-access-5vjlb\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx\" (UID: \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"
Sep 14 07:54:48.240636 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.240598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx\" (UID: \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"
Sep 14 07:54:48.240894 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.240659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f70cb1-b99f-401e-a977-6feccacf90ab-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75\" (UID: \"c7f70cb1-b99f-401e-a977-6feccacf90ab\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"
Sep 14 07:54:48.241060 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.241028    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f70cb1-b99f-401e-a977-6feccacf90ab-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75\" (UID: \"c7f70cb1-b99f-401e-a977-6feccacf90ab\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"
Sep 14 07:54:48.241209 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.241194    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f70cb1-b99f-401e-a977-6feccacf90ab-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75\" (UID: \"c7f70cb1-b99f-401e-a977-6feccacf90ab\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"
Sep 14 07:54:48.249238 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.249218    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj575\" (UniqueName: \"kubernetes.io/projected/c7f70cb1-b99f-401e-a977-6feccacf90ab-kube-api-access-mj575\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75\" (UID: \"c7f70cb1-b99f-401e-a977-6feccacf90ab\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"
Sep 14 07:54:48.295841 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.295814    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"
Sep 14 07:54:48.341459 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.341412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx\" (UID: \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"
Sep 14 07:54:48.341623 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.341536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx\" (UID: \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"
Sep 14 07:54:48.341623 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.341603    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vjlb\" (UniqueName: \"kubernetes.io/projected/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-kube-api-access-5vjlb\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx\" (UID: \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"
Sep 14 07:54:48.342202 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.342171    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx\" (UID: \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"
Sep 14 07:54:48.342312 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.342232    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx\" (UID: \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"
Sep 14 07:54:48.352658 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.352620    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vjlb\" (UniqueName: \"kubernetes.io/projected/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-kube-api-access-5vjlb\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx\" (UID: \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"
Sep 14 07:54:48.370664 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.370628    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"
Sep 14 07:54:48.415820 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.415793    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"]
Sep 14 07:54:48.417449 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:54:48.417424    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2cbc335_cad7_4265_8522_6c7e2021ca32.slice/crio-ccb6e9aca5ff28276252d42fb1162ee3de49f0cb445dff0846fa4ea9abfbbdd0 WatchSource:0}: Error finding container ccb6e9aca5ff28276252d42fb1162ee3de49f0cb445dff0846fa4ea9abfbbdd0: Status 404 returned error can't find the container with id ccb6e9aca5ff28276252d42fb1162ee3de49f0cb445dff0846fa4ea9abfbbdd0
Sep 14 07:54:48.436183 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.436159    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"
Sep 14 07:54:48.576433 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.572160    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"]
Sep 14 07:54:48.592520 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:54:48.592487    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84e3b904_d8df_43fa_931d_3e53fff06c41.slice/crio-27884d663b1d4619ee5ff9615fe31ca6f2d23d6fa49865fc80a0d4cb5a267f04 WatchSource:0}: Error finding container 27884d663b1d4619ee5ff9615fe31ca6f2d23d6fa49865fc80a0d4cb5a267f04: Status 404 returned error can't find the container with id 27884d663b1d4619ee5ff9615fe31ca6f2d23d6fa49865fc80a0d4cb5a267f04
Sep 14 07:54:48.646206 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.646182    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"]
Sep 14 07:54:48.647356 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:54:48.647329    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7f70cb1_b99f_401e_a977_6feccacf90ab.slice/crio-4cff9a12f7c63cd5e8f05b3cf70923e54515b74d7df8ff49474a2539796170a7 WatchSource:0}: Error finding container 4cff9a12f7c63cd5e8f05b3cf70923e54515b74d7df8ff49474a2539796170a7: Status 404 returned error can't find the container with id 4cff9a12f7c63cd5e8f05b3cf70923e54515b74d7df8ff49474a2539796170a7
Sep 14 07:54:48.712550 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.712523    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"]
Sep 14 07:54:48.736755 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:54:48.736722    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a2a6d69_c3af_4e92_bb0f_013ee303ccce.slice/crio-7ff3bff2897b312073d4de18d80192036549bae816a3cb8e798a0fca8e518cc6 WatchSource:0}: Error finding container 7ff3bff2897b312073d4de18d80192036549bae816a3cb8e798a0fca8e518cc6: Status 404 returned error can't find the container with id 7ff3bff2897b312073d4de18d80192036549bae816a3cb8e798a0fca8e518cc6
Sep 14 07:54:48.842229 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.842194    2563 generic.go:356] "Generic (PLEG): container finished" podID="6a2a6d69-c3af-4e92-bb0f-013ee303ccce" containerID="f9c4b804ee0bf92408a8aef021a373924e670e5bd8618fec902f44cfbc32488a" exitCode=0
Sep 14 07:54:48.842611 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.842281    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx" event={"ID":"6a2a6d69-c3af-4e92-bb0f-013ee303ccce","Type":"ContainerDied","Data":"f9c4b804ee0bf92408a8aef021a373924e670e5bd8618fec902f44cfbc32488a"}
Sep 14 07:54:48.842611 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.842338    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx" event={"ID":"6a2a6d69-c3af-4e92-bb0f-013ee303ccce","Type":"ContainerStarted","Data":"7ff3bff2897b312073d4de18d80192036549bae816a3cb8e798a0fca8e518cc6"}
Sep 14 07:54:48.843971 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.843952    2563 generic.go:356] "Generic (PLEG): container finished" podID="c7f70cb1-b99f-401e-a977-6feccacf90ab" containerID="104abec8da2abf1b2201d9bdf69b9660350468cbbb8d78dea59252e735e3aec6" exitCode=0
Sep 14 07:54:48.844110 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.844039    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75" event={"ID":"c7f70cb1-b99f-401e-a977-6feccacf90ab","Type":"ContainerDied","Data":"104abec8da2abf1b2201d9bdf69b9660350468cbbb8d78dea59252e735e3aec6"}
Sep 14 07:54:48.844110 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.844087    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75" event={"ID":"c7f70cb1-b99f-401e-a977-6feccacf90ab","Type":"ContainerStarted","Data":"4cff9a12f7c63cd5e8f05b3cf70923e54515b74d7df8ff49474a2539796170a7"}
Sep 14 07:54:48.845624 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.845602    2563 generic.go:356] "Generic (PLEG): container finished" podID="f2cbc335-cad7-4265-8522-6c7e2021ca32" containerID="2d2769d328b7745f3d97457c8bbe24d72e06388d727556be6080f03bd179c9d7" exitCode=0
Sep 14 07:54:48.845755 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.845680    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z" event={"ID":"f2cbc335-cad7-4265-8522-6c7e2021ca32","Type":"ContainerDied","Data":"2d2769d328b7745f3d97457c8bbe24d72e06388d727556be6080f03bd179c9d7"}
Sep 14 07:54:48.845755 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.845730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z" event={"ID":"f2cbc335-cad7-4265-8522-6c7e2021ca32","Type":"ContainerStarted","Data":"ccb6e9aca5ff28276252d42fb1162ee3de49f0cb445dff0846fa4ea9abfbbdd0"}
Sep 14 07:54:48.847516 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.847499    2563 generic.go:356] "Generic (PLEG): container finished" podID="84e3b904-d8df-43fa-931d-3e53fff06c41" containerID="e8f36d4ba2c3f86b0380ec7b6f7117548770127b58f540e6f2ffb89da561c3d3" exitCode=0
Sep 14 07:54:48.847608 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.847582    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4" event={"ID":"84e3b904-d8df-43fa-931d-3e53fff06c41","Type":"ContainerDied","Data":"e8f36d4ba2c3f86b0380ec7b6f7117548770127b58f540e6f2ffb89da561c3d3"}
Sep 14 07:54:48.847710 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:48.847619    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4" event={"ID":"84e3b904-d8df-43fa-931d-3e53fff06c41","Type":"ContainerStarted","Data":"27884d663b1d4619ee5ff9615fe31ca6f2d23d6fa49865fc80a0d4cb5a267f04"}
Sep 14 07:54:49.854476 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:49.854440    2563 generic.go:356] "Generic (PLEG): container finished" podID="6a2a6d69-c3af-4e92-bb0f-013ee303ccce" containerID="925fb12ce6df7853893ac9d64f6fbcfa3445f45238045390f83b90f1c792a5ca" exitCode=0
Sep 14 07:54:49.854897 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:49.854529    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx" event={"ID":"6a2a6d69-c3af-4e92-bb0f-013ee303ccce","Type":"ContainerDied","Data":"925fb12ce6df7853893ac9d64f6fbcfa3445f45238045390f83b90f1c792a5ca"}
Sep 14 07:54:49.857539 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:49.856676    2563 generic.go:356] "Generic (PLEG): container finished" podID="c7f70cb1-b99f-401e-a977-6feccacf90ab" containerID="06f54fc3ec0bf87dee6790eea3ccd43511e11978b3385d636da9363721d89cab" exitCode=0
Sep 14 07:54:49.857539 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:49.856740    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75" event={"ID":"c7f70cb1-b99f-401e-a977-6feccacf90ab","Type":"ContainerDied","Data":"06f54fc3ec0bf87dee6790eea3ccd43511e11978b3385d636da9363721d89cab"}
Sep 14 07:54:50.868557 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:50.868521    2563 generic.go:356] "Generic (PLEG): container finished" podID="6a2a6d69-c3af-4e92-bb0f-013ee303ccce" containerID="1910a39acd6c13592fd8e4fe67b53e9eb5b8aa5445054809d4e8b282edeae151" exitCode=0
Sep 14 07:54:50.868996 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:50.868593    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx" event={"ID":"6a2a6d69-c3af-4e92-bb0f-013ee303ccce","Type":"ContainerDied","Data":"1910a39acd6c13592fd8e4fe67b53e9eb5b8aa5445054809d4e8b282edeae151"}
Sep 14 07:54:50.870636 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:50.870610    2563 generic.go:356] "Generic (PLEG): container finished" podID="c7f70cb1-b99f-401e-a977-6feccacf90ab" containerID="54950fd82b398a8049fb541e368a58baa4284952c885144f289ab580005d2a07" exitCode=0
Sep 14 07:54:50.870779 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:50.870695    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75" event={"ID":"c7f70cb1-b99f-401e-a977-6feccacf90ab","Type":"ContainerDied","Data":"54950fd82b398a8049fb541e368a58baa4284952c885144f289ab580005d2a07"}
Sep 14 07:54:50.872262 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:50.872242    2563 generic.go:356] "Generic (PLEG): container finished" podID="f2cbc335-cad7-4265-8522-6c7e2021ca32" containerID="b0a5a45ae2906473c5bc920d89fe2acf8924a41b0a7cd46809308b5cedb7417c" exitCode=0
Sep 14 07:54:50.872348 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:50.872317    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z" event={"ID":"f2cbc335-cad7-4265-8522-6c7e2021ca32","Type":"ContainerDied","Data":"b0a5a45ae2906473c5bc920d89fe2acf8924a41b0a7cd46809308b5cedb7417c"}
Sep 14 07:54:50.874327 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:50.874307    2563 generic.go:356] "Generic (PLEG): container finished" podID="84e3b904-d8df-43fa-931d-3e53fff06c41" containerID="b0036ee5e3ea3f8e74e07316611e7b3cc9050fc50ed053d6a07ea70c9ef25145" exitCode=0
Sep 14 07:54:50.874400 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:50.874356    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4" event={"ID":"84e3b904-d8df-43fa-931d-3e53fff06c41","Type":"ContainerDied","Data":"b0036ee5e3ea3f8e74e07316611e7b3cc9050fc50ed053d6a07ea70c9ef25145"}
Sep 14 07:54:51.881933 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:51.881901    2563 generic.go:356] "Generic (PLEG): container finished" podID="f2cbc335-cad7-4265-8522-6c7e2021ca32" containerID="29cddc6bd5c890cc486f1ac02372745345479fba096cff1377e78e8627551c5a" exitCode=0
Sep 14 07:54:51.884257 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:51.884232    2563 generic.go:356] "Generic (PLEG): container finished" podID="84e3b904-d8df-43fa-931d-3e53fff06c41" containerID="749bdea9689f989e44345fcdfb6fec2ffd34c878479ce3b11812214679395d1e" exitCode=0
Sep 14 07:54:51.885911 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:51.885887    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z" event={"ID":"f2cbc335-cad7-4265-8522-6c7e2021ca32","Type":"ContainerDied","Data":"29cddc6bd5c890cc486f1ac02372745345479fba096cff1377e78e8627551c5a"}
Sep 14 07:54:51.885981 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:51.885928    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4" event={"ID":"84e3b904-d8df-43fa-931d-3e53fff06c41","Type":"ContainerDied","Data":"749bdea9689f989e44345fcdfb6fec2ffd34c878479ce3b11812214679395d1e"}
Sep 14 07:54:52.027979 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.027955    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"
Sep 14 07:54:52.047907 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.047880    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"
Sep 14 07:54:52.176357 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.176257    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-util\") pod \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\" (UID: \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\") "
Sep 14 07:54:52.176357 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.176336    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-bundle\") pod \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\" (UID: \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\") "
Sep 14 07:54:52.176539 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.176371    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f70cb1-b99f-401e-a977-6feccacf90ab-bundle\") pod \"c7f70cb1-b99f-401e-a977-6feccacf90ab\" (UID: \"c7f70cb1-b99f-401e-a977-6feccacf90ab\") "
Sep 14 07:54:52.176539 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.176411    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vjlb\" (UniqueName: \"kubernetes.io/projected/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-kube-api-access-5vjlb\") pod \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\" (UID: \"6a2a6d69-c3af-4e92-bb0f-013ee303ccce\") "
Sep 14 07:54:52.176539 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.176452    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f70cb1-b99f-401e-a977-6feccacf90ab-util\") pod \"c7f70cb1-b99f-401e-a977-6feccacf90ab\" (UID: \"c7f70cb1-b99f-401e-a977-6feccacf90ab\") "
Sep 14 07:54:52.176539 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.176491    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj575\" (UniqueName: \"kubernetes.io/projected/c7f70cb1-b99f-401e-a977-6feccacf90ab-kube-api-access-mj575\") pod \"c7f70cb1-b99f-401e-a977-6feccacf90ab\" (UID: \"c7f70cb1-b99f-401e-a977-6feccacf90ab\") "
Sep 14 07:54:52.176994 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.176971    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7f70cb1-b99f-401e-a977-6feccacf90ab-bundle" (OuterVolumeSpecName: "bundle") pod "c7f70cb1-b99f-401e-a977-6feccacf90ab" (UID: "c7f70cb1-b99f-401e-a977-6feccacf90ab"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:54:52.177158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.177131    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-bundle" (OuterVolumeSpecName: "bundle") pod "6a2a6d69-c3af-4e92-bb0f-013ee303ccce" (UID: "6a2a6d69-c3af-4e92-bb0f-013ee303ccce"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:54:52.178739 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.178697    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-kube-api-access-5vjlb" (OuterVolumeSpecName: "kube-api-access-5vjlb") pod "6a2a6d69-c3af-4e92-bb0f-013ee303ccce" (UID: "6a2a6d69-c3af-4e92-bb0f-013ee303ccce"). InnerVolumeSpecName "kube-api-access-5vjlb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:54:52.178881 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.178860    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7f70cb1-b99f-401e-a977-6feccacf90ab-kube-api-access-mj575" (OuterVolumeSpecName: "kube-api-access-mj575") pod "c7f70cb1-b99f-401e-a977-6feccacf90ab" (UID: "c7f70cb1-b99f-401e-a977-6feccacf90ab"). InnerVolumeSpecName "kube-api-access-mj575". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:54:52.182189 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.182171    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-util" (OuterVolumeSpecName: "util") pod "6a2a6d69-c3af-4e92-bb0f-013ee303ccce" (UID: "6a2a6d69-c3af-4e92-bb0f-013ee303ccce"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:54:52.184586 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.184564    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7f70cb1-b99f-401e-a977-6feccacf90ab-util" (OuterVolumeSpecName: "util") pod "c7f70cb1-b99f-401e-a977-6feccacf90ab" (UID: "c7f70cb1-b99f-401e-a977-6feccacf90ab"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:54:52.278184 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.278141    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vjlb\" (UniqueName: \"kubernetes.io/projected/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-kube-api-access-5vjlb\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:54:52.278184 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.278181    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7f70cb1-b99f-401e-a977-6feccacf90ab-util\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:54:52.278184 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.278192    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj575\" (UniqueName: \"kubernetes.io/projected/c7f70cb1-b99f-401e-a977-6feccacf90ab-kube-api-access-mj575\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:54:52.278184 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.278202    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-util\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:54:52.278425 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.278211    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a2a6d69-c3af-4e92-bb0f-013ee303ccce-bundle\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:54:52.278425 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.278219    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7f70cb1-b99f-401e-a977-6feccacf90ab-bundle\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:54:52.891110 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.891079    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx" event={"ID":"6a2a6d69-c3af-4e92-bb0f-013ee303ccce","Type":"ContainerDied","Data":"7ff3bff2897b312073d4de18d80192036549bae816a3cb8e798a0fca8e518cc6"}
Sep 14 07:54:52.891110 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.891100    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an8dzx"
Sep 14 07:54:52.891582 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.891126    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ff3bff2897b312073d4de18d80192036549bae816a3cb8e798a0fca8e518cc6"
Sep 14 07:54:52.892900 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.892876    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75" event={"ID":"c7f70cb1-b99f-401e-a977-6feccacf90ab","Type":"ContainerDied","Data":"4cff9a12f7c63cd5e8f05b3cf70923e54515b74d7df8ff49474a2539796170a7"}
Sep 14 07:54:52.893024 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.892910    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d798g75"
Sep 14 07:54:52.893024 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:52.892916    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cff9a12f7c63cd5e8f05b3cf70923e54515b74d7df8ff49474a2539796170a7"
Sep 14 07:54:53.035244 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.035220    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"
Sep 14 07:54:53.065035 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.065012    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"
Sep 14 07:54:53.186826 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.186738    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2cbc335-cad7-4265-8522-6c7e2021ca32-util\") pod \"f2cbc335-cad7-4265-8522-6c7e2021ca32\" (UID: \"f2cbc335-cad7-4265-8522-6c7e2021ca32\") "
Sep 14 07:54:53.186826 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.186806    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krmh8\" (UniqueName: \"kubernetes.io/projected/f2cbc335-cad7-4265-8522-6c7e2021ca32-kube-api-access-krmh8\") pod \"f2cbc335-cad7-4265-8522-6c7e2021ca32\" (UID: \"f2cbc335-cad7-4265-8522-6c7e2021ca32\") "
Sep 14 07:54:53.187018 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.186836    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84e3b904-d8df-43fa-931d-3e53fff06c41-bundle\") pod \"84e3b904-d8df-43fa-931d-3e53fff06c41\" (UID: \"84e3b904-d8df-43fa-931d-3e53fff06c41\") "
Sep 14 07:54:53.187018 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.186892    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2cbc335-cad7-4265-8522-6c7e2021ca32-bundle\") pod \"f2cbc335-cad7-4265-8522-6c7e2021ca32\" (UID: \"f2cbc335-cad7-4265-8522-6c7e2021ca32\") "
Sep 14 07:54:53.187018 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.186929    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84e3b904-d8df-43fa-931d-3e53fff06c41-util\") pod \"84e3b904-d8df-43fa-931d-3e53fff06c41\" (UID: \"84e3b904-d8df-43fa-931d-3e53fff06c41\") "
Sep 14 07:54:53.187018 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.186954    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxt4g\" (UniqueName: \"kubernetes.io/projected/84e3b904-d8df-43fa-931d-3e53fff06c41-kube-api-access-qxt4g\") pod \"84e3b904-d8df-43fa-931d-3e53fff06c41\" (UID: \"84e3b904-d8df-43fa-931d-3e53fff06c41\") "
Sep 14 07:54:53.187476 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.187444    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84e3b904-d8df-43fa-931d-3e53fff06c41-bundle" (OuterVolumeSpecName: "bundle") pod "84e3b904-d8df-43fa-931d-3e53fff06c41" (UID: "84e3b904-d8df-43fa-931d-3e53fff06c41"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:54:53.187595 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.187518    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2cbc335-cad7-4265-8522-6c7e2021ca32-bundle" (OuterVolumeSpecName: "bundle") pod "f2cbc335-cad7-4265-8522-6c7e2021ca32" (UID: "f2cbc335-cad7-4265-8522-6c7e2021ca32"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:54:53.189095 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.189073    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2cbc335-cad7-4265-8522-6c7e2021ca32-kube-api-access-krmh8" (OuterVolumeSpecName: "kube-api-access-krmh8") pod "f2cbc335-cad7-4265-8522-6c7e2021ca32" (UID: "f2cbc335-cad7-4265-8522-6c7e2021ca32"). InnerVolumeSpecName "kube-api-access-krmh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:54:53.189202 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.189092    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84e3b904-d8df-43fa-931d-3e53fff06c41-kube-api-access-qxt4g" (OuterVolumeSpecName: "kube-api-access-qxt4g") pod "84e3b904-d8df-43fa-931d-3e53fff06c41" (UID: "84e3b904-d8df-43fa-931d-3e53fff06c41"). InnerVolumeSpecName "kube-api-access-qxt4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:54:53.195862 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.195837    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2cbc335-cad7-4265-8522-6c7e2021ca32-util" (OuterVolumeSpecName: "util") pod "f2cbc335-cad7-4265-8522-6c7e2021ca32" (UID: "f2cbc335-cad7-4265-8522-6c7e2021ca32"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:54:53.196123 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.196108    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84e3b904-d8df-43fa-931d-3e53fff06c41-util" (OuterVolumeSpecName: "util") pod "84e3b904-d8df-43fa-931d-3e53fff06c41" (UID: "84e3b904-d8df-43fa-931d-3e53fff06c41"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:54:53.288169 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.288143    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krmh8\" (UniqueName: \"kubernetes.io/projected/f2cbc335-cad7-4265-8522-6c7e2021ca32-kube-api-access-krmh8\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:54:53.288169 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.288166    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84e3b904-d8df-43fa-931d-3e53fff06c41-bundle\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:54:53.288321 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.288177    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2cbc335-cad7-4265-8522-6c7e2021ca32-bundle\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:54:53.288321 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.288185    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84e3b904-d8df-43fa-931d-3e53fff06c41-util\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:54:53.288321 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.288192    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxt4g\" (UniqueName: \"kubernetes.io/projected/84e3b904-d8df-43fa-931d-3e53fff06c41-kube-api-access-qxt4g\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:54:53.288321 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.288200    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2cbc335-cad7-4265-8522-6c7e2021ca32-util\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:54:53.899522 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.899497    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4"
Sep 14 07:54:53.899876 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.899496    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67gnb4" event={"ID":"84e3b904-d8df-43fa-931d-3e53fff06c41","Type":"ContainerDied","Data":"27884d663b1d4619ee5ff9615fe31ca6f2d23d6fa49865fc80a0d4cb5a267f04"}
Sep 14 07:54:53.899876 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.899611    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27884d663b1d4619ee5ff9615fe31ca6f2d23d6fa49865fc80a0d4cb5a267f04"
Sep 14 07:54:53.901696 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.901669    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z" event={"ID":"f2cbc335-cad7-4265-8522-6c7e2021ca32","Type":"ContainerDied","Data":"ccb6e9aca5ff28276252d42fb1162ee3de49f0cb445dff0846fa4ea9abfbbdd0"}
Sep 14 07:54:53.901790 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.901710    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ccb6e9aca5ff28276252d42fb1162ee3de49f0cb445dff0846fa4ea9abfbbdd0"
Sep 14 07:54:53.901825 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:53.901793    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4chgm7z"
Sep 14 07:54:54.484617 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:54.484584    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:54.484841 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:54.484663    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:54.489499 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:54.489474    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:54.910477 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:54.910451    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7679fc7b7c-hhqbr"
Sep 14 07:54:59.839178 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839144    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"]
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839523    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e3b904-d8df-43fa-931d-3e53fff06c41" containerName="pull"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839543    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e3b904-d8df-43fa-931d-3e53fff06c41" containerName="pull"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839561    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a2a6d69-c3af-4e92-bb0f-013ee303ccce" containerName="util"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839568    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a2a6d69-c3af-4e92-bb0f-013ee303ccce" containerName="util"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839578    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2cbc335-cad7-4265-8522-6c7e2021ca32" containerName="util"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839586    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2cbc335-cad7-4265-8522-6c7e2021ca32" containerName="util"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839595    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2cbc335-cad7-4265-8522-6c7e2021ca32" containerName="pull"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839603    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2cbc335-cad7-4265-8522-6c7e2021ca32" containerName="pull"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839612    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e3b904-d8df-43fa-931d-3e53fff06c41" containerName="extract"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839620    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e3b904-d8df-43fa-931d-3e53fff06c41" containerName="extract"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839633    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f70cb1-b99f-401e-a977-6feccacf90ab" containerName="pull"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839661    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f70cb1-b99f-401e-a977-6feccacf90ab" containerName="pull"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839675    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f70cb1-b99f-401e-a977-6feccacf90ab" containerName="extract"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839680    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f70cb1-b99f-401e-a977-6feccacf90ab" containerName="extract"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839691    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a2a6d69-c3af-4e92-bb0f-013ee303ccce" containerName="pull"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839696    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a2a6d69-c3af-4e92-bb0f-013ee303ccce" containerName="pull"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839703    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f70cb1-b99f-401e-a977-6feccacf90ab" containerName="util"
Sep 14 07:54:59.839701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839708    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f70cb1-b99f-401e-a977-6feccacf90ab" containerName="util"
Sep 14 07:54:59.840266 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839715    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e3b904-d8df-43fa-931d-3e53fff06c41" containerName="util"
Sep 14 07:54:59.840266 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839721    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e3b904-d8df-43fa-931d-3e53fff06c41" containerName="util"
Sep 14 07:54:59.840266 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839726    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2cbc335-cad7-4265-8522-6c7e2021ca32" containerName="extract"
Sep 14 07:54:59.840266 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839733    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2cbc335-cad7-4265-8522-6c7e2021ca32" containerName="extract"
Sep 14 07:54:59.840266 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839748    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a2a6d69-c3af-4e92-bb0f-013ee303ccce" containerName="extract"
Sep 14 07:54:59.840266 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839754    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a2a6d69-c3af-4e92-bb0f-013ee303ccce" containerName="extract"
Sep 14 07:54:59.840266 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839843    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7f70cb1-b99f-401e-a977-6feccacf90ab" containerName="extract"
Sep 14 07:54:59.840266 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839857    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="84e3b904-d8df-43fa-931d-3e53fff06c41" containerName="extract"
Sep 14 07:54:59.840266 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839866    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2cbc335-cad7-4265-8522-6c7e2021ca32" containerName="extract"
Sep 14 07:54:59.840266 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.839875    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a2a6d69-c3af-4e92-bb0f-013ee303ccce" containerName="extract"
Sep 14 07:54:59.843627 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.843612    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"
Sep 14 07:54:59.848313 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.848292    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 14 07:54:59.848901 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.848879    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 14 07:54:59.849019 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.848888    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-vdvfj\""
Sep 14 07:54:59.862736 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.862702    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"]
Sep 14 07:54:59.946675 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:54:59.946630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc969\" (UniqueName: \"kubernetes.io/projected/f4355e24-6e22-4e2b-9eaf-81aabb1ea367-kube-api-access-mc969\") pod \"limitador-operator-controller-manager-74495c69f8-f9qq6\" (UID: \"f4355e24-6e22-4e2b-9eaf-81aabb1ea367\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"
Sep 14 07:55:00.047334 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:00.047302    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc969\" (UniqueName: \"kubernetes.io/projected/f4355e24-6e22-4e2b-9eaf-81aabb1ea367-kube-api-access-mc969\") pod \"limitador-operator-controller-manager-74495c69f8-f9qq6\" (UID: \"f4355e24-6e22-4e2b-9eaf-81aabb1ea367\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"
Sep 14 07:55:00.060677 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:00.060652    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc969\" (UniqueName: \"kubernetes.io/projected/f4355e24-6e22-4e2b-9eaf-81aabb1ea367-kube-api-access-mc969\") pod \"limitador-operator-controller-manager-74495c69f8-f9qq6\" (UID: \"f4355e24-6e22-4e2b-9eaf-81aabb1ea367\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"
Sep 14 07:55:00.155057 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:00.154972    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"
Sep 14 07:55:00.446681 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:00.446655    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"]
Sep 14 07:55:00.448803 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:55:00.448775    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4355e24_6e22_4e2b_9eaf_81aabb1ea367.slice/crio-64a5df8bcab6d627a2e7ef313e63da4409664c7a76e77e86e126639c150dacf6 WatchSource:0}: Error finding container 64a5df8bcab6d627a2e7ef313e63da4409664c7a76e77e86e126639c150dacf6: Status 404 returned error can't find the container with id 64a5df8bcab6d627a2e7ef313e63da4409664c7a76e77e86e126639c150dacf6
Sep 14 07:55:00.935427 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:00.935390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6" event={"ID":"f4355e24-6e22-4e2b-9eaf-81aabb1ea367","Type":"ContainerStarted","Data":"64a5df8bcab6d627a2e7ef313e63da4409664c7a76e77e86e126639c150dacf6"}
Sep 14 07:55:03.954291 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:03.954251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6" event={"ID":"f4355e24-6e22-4e2b-9eaf-81aabb1ea367","Type":"ContainerStarted","Data":"e98a7cbd0fbeb39f46f78cc6936f073c268de37d79a92f37a5c66a89736e0d12"}
Sep 14 07:55:03.954704 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:03.954338    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"
Sep 14 07:55:06.909688 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:06.909604    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6" podStartSLOduration=5.444266025 podStartE2EDuration="7.909586576s" totalImagesPullingTime="2.465320551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:54:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:03.987022028 +0000 UTC m=+424.615909655" watchObservedRunningTime="2026-09-14 07:55:06.909586576 +0000 UTC m=+427.538474202"
Sep 14 07:55:06.911628 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:06.911605    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"]
Sep 14 07:55:06.915062 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:06.915045    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:06.918456 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:06.918427    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xpfct\""
Sep 14 07:55:06.948512 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:06.948479    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"]
Sep 14 07:55:07.010340 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:07.010297    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mtnt\" (UniqueName: \"kubernetes.io/projected/1b6bb72c-6cc6-4500-b931-9d10dfdb1481-kube-api-access-2mtnt\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k2zwh\" (UID: \"1b6bb72c-6cc6-4500-b931-9d10dfdb1481\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:07.010531 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:07.010451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b6bb72c-6cc6-4500-b931-9d10dfdb1481-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k2zwh\" (UID: \"1b6bb72c-6cc6-4500-b931-9d10dfdb1481\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:07.110906 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:07.110861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b6bb72c-6cc6-4500-b931-9d10dfdb1481-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k2zwh\" (UID: \"1b6bb72c-6cc6-4500-b931-9d10dfdb1481\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:07.110906 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:07.110910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mtnt\" (UniqueName: \"kubernetes.io/projected/1b6bb72c-6cc6-4500-b931-9d10dfdb1481-kube-api-access-2mtnt\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k2zwh\" (UID: \"1b6bb72c-6cc6-4500-b931-9d10dfdb1481\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:07.111269 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:07.111247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b6bb72c-6cc6-4500-b931-9d10dfdb1481-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k2zwh\" (UID: \"1b6bb72c-6cc6-4500-b931-9d10dfdb1481\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:07.122388 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:07.122354    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mtnt\" (UniqueName: \"kubernetes.io/projected/1b6bb72c-6cc6-4500-b931-9d10dfdb1481-kube-api-access-2mtnt\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k2zwh\" (UID: \"1b6bb72c-6cc6-4500-b931-9d10dfdb1481\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:07.224992 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:07.224903    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:07.507523 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:07.507489    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"]
Sep 14 07:55:07.508587 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:55:07.508561    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b6bb72c_6cc6_4500_b931_9d10dfdb1481.slice/crio-7593843442b33d97bec486c43db664be125ea32e62c3b2b6e4ebc0792556babb WatchSource:0}: Error finding container 7593843442b33d97bec486c43db664be125ea32e62c3b2b6e4ebc0792556babb: Status 404 returned error can't find the container with id 7593843442b33d97bec486c43db664be125ea32e62c3b2b6e4ebc0792556babb
Sep 14 07:55:07.977661 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:07.977609    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh" event={"ID":"1b6bb72c-6cc6-4500-b931-9d10dfdb1481","Type":"ContainerStarted","Data":"7593843442b33d97bec486c43db664be125ea32e62c3b2b6e4ebc0792556babb"}
Sep 14 07:55:12.002000 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:12.001958    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh" event={"ID":"1b6bb72c-6cc6-4500-b931-9d10dfdb1481","Type":"ContainerStarted","Data":"f5a135dca16728954775584e9589dfa39e37ab9e881b5be8aefcd6c70b2cd4fe"}
Sep 14 07:55:12.002480 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:12.002046    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:12.024621 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:12.024573    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh" podStartSLOduration=2.245869809 podStartE2EDuration="6.024559377s" totalImagesPullingTime="3.778689568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:12.021820374 +0000 UTC m=+432.650708013" watchObservedRunningTime="2026-09-14 07:55:12.024559377 +0000 UTC m=+432.653447003"
Sep 14 07:55:14.964958 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:14.964924    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"
Sep 14 07:55:23.010482 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:23.010396    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:24.619208 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:24.619175    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"]
Sep 14 07:55:24.622526 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:24.622511    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:24.634213 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:24.634188    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"]
Sep 14 07:55:24.665251 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:24.665218    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/922842b7-6334-48e4-b6c4-c23fcce6593e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-jhcsb\" (UID: \"922842b7-6334-48e4-b6c4-c23fcce6593e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:24.665383 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:24.665259    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlk2b\" (UniqueName: \"kubernetes.io/projected/922842b7-6334-48e4-b6c4-c23fcce6593e-kube-api-access-vlk2b\") pod \"kuadrant-operator-controller-manager-5f685c496f-jhcsb\" (UID: \"922842b7-6334-48e4-b6c4-c23fcce6593e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:24.766588 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:24.766551    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/922842b7-6334-48e4-b6c4-c23fcce6593e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-jhcsb\" (UID: \"922842b7-6334-48e4-b6c4-c23fcce6593e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:24.766777 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:24.766606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlk2b\" (UniqueName: \"kubernetes.io/projected/922842b7-6334-48e4-b6c4-c23fcce6593e-kube-api-access-vlk2b\") pod \"kuadrant-operator-controller-manager-5f685c496f-jhcsb\" (UID: \"922842b7-6334-48e4-b6c4-c23fcce6593e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:24.766951 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:24.766934    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/922842b7-6334-48e4-b6c4-c23fcce6593e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-jhcsb\" (UID: \"922842b7-6334-48e4-b6c4-c23fcce6593e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:24.778183 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:24.778151    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlk2b\" (UniqueName: \"kubernetes.io/projected/922842b7-6334-48e4-b6c4-c23fcce6593e-kube-api-access-vlk2b\") pod \"kuadrant-operator-controller-manager-5f685c496f-jhcsb\" (UID: \"922842b7-6334-48e4-b6c4-c23fcce6593e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:24.932888 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:24.932804    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:25.211950 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.211922    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"]
Sep 14 07:55:25.213372 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:55:25.213347    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922842b7_6334_48e4_b6c4_c23fcce6593e.slice/crio-1965034c497d191b44f1daf1e3e5beca0a80b1b3f25fe408df36226e1c7a991c WatchSource:0}: Error finding container 1965034c497d191b44f1daf1e3e5beca0a80b1b3f25fe408df36226e1c7a991c: Status 404 returned error can't find the container with id 1965034c497d191b44f1daf1e3e5beca0a80b1b3f25fe408df36226e1c7a991c
Sep 14 07:55:25.447001 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.446967    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"]
Sep 14 07:55:25.451421 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.451397    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"]
Sep 14 07:55:25.475076 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.474994    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"]
Sep 14 07:55:25.475404 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.475386    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="922842b7-6334-48e4-b6c4-c23fcce6593e" containerName="manager"
Sep 14 07:55:25.475488 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.475406    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="922842b7-6334-48e4-b6c4-c23fcce6593e" containerName="manager"
Sep 14 07:55:25.475542 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.475527    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="922842b7-6334-48e4-b6c4-c23fcce6593e" containerName="manager"
Sep 14 07:55:25.478522 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.478500    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"
Sep 14 07:55:25.495498 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.495472    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"]
Sep 14 07:55:25.495704 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.495661    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh" podUID="1b6bb72c-6cc6-4500-b931-9d10dfdb1481" containerName="manager" containerID="cri-o://f5a135dca16728954775584e9589dfa39e37ab9e881b5be8aefcd6c70b2cd4fe" gracePeriod=2
Sep 14 07:55:25.497701 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.497673    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"]
Sep 14 07:55:25.507139 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.507117    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"]
Sep 14 07:55:25.510881 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:25.510851    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-k2zwh\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="1b6bb72c-6cc6-4500-b931-9d10dfdb1481" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:25.521135 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.521109    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"]
Sep 14 07:55:25.521306 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.521283    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6" podUID="f4355e24-6e22-4e2b-9eaf-81aabb1ea367" containerName="manager" containerID="cri-o://e98a7cbd0fbeb39f46f78cc6936f073c268de37d79a92f37a5c66a89736e0d12" gracePeriod=2
Sep 14 07:55:25.539097 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.539067    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"]
Sep 14 07:55:25.544135 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:25.544091    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f4355e24-6e22-4e2b-9eaf-81aabb1ea367\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-f9qq6\": pods \"limitador-operator-controller-manager-74495c69f8-f9qq6\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"
Sep 14 07:55:25.545975 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:25.545938    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-k2zwh\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="1b6bb72c-6cc6-4500-b931-9d10dfdb1481" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:25.547557 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.547533    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-hk89x"]
Sep 14 07:55:25.547943 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.547925    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4355e24-6e22-4e2b-9eaf-81aabb1ea367" containerName="manager"
Sep 14 07:55:25.548024 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.547945    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4355e24-6e22-4e2b-9eaf-81aabb1ea367" containerName="manager"
Sep 14 07:55:25.548024 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.547957    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b6bb72c-6cc6-4500-b931-9d10dfdb1481" containerName="manager"
Sep 14 07:55:25.548024 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.547965    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b6bb72c-6cc6-4500-b931-9d10dfdb1481" containerName="manager"
Sep 14 07:55:25.548163 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.548065    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4355e24-6e22-4e2b-9eaf-81aabb1ea367" containerName="manager"
Sep 14 07:55:25.548163 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.548076    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b6bb72c-6cc6-4500-b931-9d10dfdb1481" containerName="manager"
Sep 14 07:55:25.551038 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.551022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hk89x"
Sep 14 07:55:25.553416 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:25.553381    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-f9qq6\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="f4355e24-6e22-4e2b-9eaf-81aabb1ea367" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"
Sep 14 07:55:25.554900 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:25.554876    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-k2zwh\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="1b6bb72c-6cc6-4500-b931-9d10dfdb1481" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:25.564586 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.564560    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-hk89x"]
Sep 14 07:55:25.574323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.574040    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76d78b20-acce-4d41-bc23-6a17653a0a90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-hktdk\" (UID: \"76d78b20-acce-4d41-bc23-6a17653a0a90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"
Sep 14 07:55:25.574323 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.574144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2lgv\" (UniqueName: \"kubernetes.io/projected/76d78b20-acce-4d41-bc23-6a17653a0a90-kube-api-access-g2lgv\") pod \"kuadrant-operator-controller-manager-5f685c496f-hktdk\" (UID: \"76d78b20-acce-4d41-bc23-6a17653a0a90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"
Sep 14 07:55:25.675346 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.675308    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm9g4\" (UniqueName: \"kubernetes.io/projected/43ebfe24-78eb-4c7d-a77c-4ea96d27269a-kube-api-access-bm9g4\") pod \"limitador-operator-controller-manager-74495c69f8-hk89x\" (UID: \"43ebfe24-78eb-4c7d-a77c-4ea96d27269a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hk89x"
Sep 14 07:55:25.675783 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.675359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2lgv\" (UniqueName: \"kubernetes.io/projected/76d78b20-acce-4d41-bc23-6a17653a0a90-kube-api-access-g2lgv\") pod \"kuadrant-operator-controller-manager-5f685c496f-hktdk\" (UID: \"76d78b20-acce-4d41-bc23-6a17653a0a90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"
Sep 14 07:55:25.675783 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.675503    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76d78b20-acce-4d41-bc23-6a17653a0a90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-hktdk\" (UID: \"76d78b20-acce-4d41-bc23-6a17653a0a90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"
Sep 14 07:55:25.675914 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.675896    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76d78b20-acce-4d41-bc23-6a17653a0a90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-hktdk\" (UID: \"76d78b20-acce-4d41-bc23-6a17653a0a90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"
Sep 14 07:55:25.684766 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.684719    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2lgv\" (UniqueName: \"kubernetes.io/projected/76d78b20-acce-4d41-bc23-6a17653a0a90-kube-api-access-g2lgv\") pod \"kuadrant-operator-controller-manager-5f685c496f-hktdk\" (UID: \"76d78b20-acce-4d41-bc23-6a17653a0a90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"
Sep 14 07:55:25.750984 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.750956    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:25.752856 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:25.752830    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-f9qq6\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="f4355e24-6e22-4e2b-9eaf-81aabb1ea367" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"
Sep 14 07:55:25.754245 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:25.754223    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-k2zwh\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="1b6bb72c-6cc6-4500-b931-9d10dfdb1481" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:25.764045 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.764022    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"
Sep 14 07:55:25.766111 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:25.766078    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-f9qq6\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="f4355e24-6e22-4e2b-9eaf-81aabb1ea367" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"
Sep 14 07:55:25.767687 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:25.767659    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-k2zwh\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="1b6bb72c-6cc6-4500-b931-9d10dfdb1481" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:25.776187 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.776165    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm9g4\" (UniqueName: \"kubernetes.io/projected/43ebfe24-78eb-4c7d-a77c-4ea96d27269a-kube-api-access-bm9g4\") pod \"limitador-operator-controller-manager-74495c69f8-hk89x\" (UID: \"43ebfe24-78eb-4c7d-a77c-4ea96d27269a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hk89x"
Sep 14 07:55:25.789054 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.789034    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"
Sep 14 07:55:25.792026 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.792008    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm9g4\" (UniqueName: \"kubernetes.io/projected/43ebfe24-78eb-4c7d-a77c-4ea96d27269a-kube-api-access-bm9g4\") pod \"limitador-operator-controller-manager-74495c69f8-hk89x\" (UID: \"43ebfe24-78eb-4c7d-a77c-4ea96d27269a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hk89x"
Sep 14 07:55:25.876653 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.876613    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mc969\" (UniqueName: \"kubernetes.io/projected/f4355e24-6e22-4e2b-9eaf-81aabb1ea367-kube-api-access-mc969\") pod \"f4355e24-6e22-4e2b-9eaf-81aabb1ea367\" (UID: \"f4355e24-6e22-4e2b-9eaf-81aabb1ea367\") "
Sep 14 07:55:25.876827 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.876680    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mtnt\" (UniqueName: \"kubernetes.io/projected/1b6bb72c-6cc6-4500-b931-9d10dfdb1481-kube-api-access-2mtnt\") pod \"1b6bb72c-6cc6-4500-b931-9d10dfdb1481\" (UID: \"1b6bb72c-6cc6-4500-b931-9d10dfdb1481\") "
Sep 14 07:55:25.876827 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.876723    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b6bb72c-6cc6-4500-b931-9d10dfdb1481-extensions-socket-volume\") pod \"1b6bb72c-6cc6-4500-b931-9d10dfdb1481\" (UID: \"1b6bb72c-6cc6-4500-b931-9d10dfdb1481\") "
Sep 14 07:55:25.877380 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.877330    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b6bb72c-6cc6-4500-b931-9d10dfdb1481-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1b6bb72c-6cc6-4500-b931-9d10dfdb1481" (UID: "1b6bb72c-6cc6-4500-b931-9d10dfdb1481"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:55:25.878897 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.878869    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4355e24-6e22-4e2b-9eaf-81aabb1ea367-kube-api-access-mc969" (OuterVolumeSpecName: "kube-api-access-mc969") pod "f4355e24-6e22-4e2b-9eaf-81aabb1ea367" (UID: "f4355e24-6e22-4e2b-9eaf-81aabb1ea367"). InnerVolumeSpecName "kube-api-access-mc969". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:55:25.879035 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.878903    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b6bb72c-6cc6-4500-b931-9d10dfdb1481-kube-api-access-2mtnt" (OuterVolumeSpecName: "kube-api-access-2mtnt") pod "1b6bb72c-6cc6-4500-b931-9d10dfdb1481" (UID: "1b6bb72c-6cc6-4500-b931-9d10dfdb1481"). InnerVolumeSpecName "kube-api-access-2mtnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:55:25.886093 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.886068    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b6bb72c-6cc6-4500-b931-9d10dfdb1481" path="/var/lib/kubelet/pods/1b6bb72c-6cc6-4500-b931-9d10dfdb1481/volumes"
Sep 14 07:55:25.886398 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.886386    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4355e24-6e22-4e2b-9eaf-81aabb1ea367" path="/var/lib/kubelet/pods/f4355e24-6e22-4e2b-9eaf-81aabb1ea367/volumes"
Sep 14 07:55:25.919117 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.919088    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hk89x"
Sep 14 07:55:25.978544 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.978517    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b6bb72c-6cc6-4500-b931-9d10dfdb1481-extensions-socket-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:55:25.978544 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.978545    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mc969\" (UniqueName: \"kubernetes.io/projected/f4355e24-6e22-4e2b-9eaf-81aabb1ea367-kube-api-access-mc969\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:55:25.978731 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:25.978554    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mtnt\" (UniqueName: \"kubernetes.io/projected/1b6bb72c-6cc6-4500-b931-9d10dfdb1481-kube-api-access-2mtnt\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:55:26.081548 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:26.081526    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb" container="manager"
Sep 14 07:55:26.082492 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:26.082467    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"]
Sep 14 07:55:26.083054 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:26.083025    2563 generic.go:356] "Generic (PLEG): container finished" podID="f4355e24-6e22-4e2b-9eaf-81aabb1ea367" containerID="e98a7cbd0fbeb39f46f78cc6936f073c268de37d79a92f37a5c66a89736e0d12" exitCode=0
Sep 14 07:55:26.083153 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:26.083061    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-f9qq6"
Sep 14 07:55:26.083153 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:26.083123    2563 scope.go:117] "RemoveContainer" containerID="e98a7cbd0fbeb39f46f78cc6936f073c268de37d79a92f37a5c66a89736e0d12"
Sep 14 07:55:26.084082 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:55:26.084057    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d78b20_acce_4d41_bc23_6a17653a0a90.slice/crio-0b67964a87d76fcac3112bffd34f4eb81db81998978ff1f9315908c57f1774a9 WatchSource:0}: Error finding container 0b67964a87d76fcac3112bffd34f4eb81db81998978ff1f9315908c57f1774a9: Status 404 returned error can't find the container with id 0b67964a87d76fcac3112bffd34f4eb81db81998978ff1f9315908c57f1774a9
Sep 14 07:55:26.084809 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:26.084784    2563 generic.go:356] "Generic (PLEG): container finished" podID="1b6bb72c-6cc6-4500-b931-9d10dfdb1481" containerID="f5a135dca16728954775584e9589dfa39e37ab9e881b5be8aefcd6c70b2cd4fe" exitCode=0
Sep 14 07:55:26.084913 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:26.084838    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k2zwh"
Sep 14 07:55:26.092920 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:26.092903    2563 scope.go:117] "RemoveContainer" containerID="e98a7cbd0fbeb39f46f78cc6936f073c268de37d79a92f37a5c66a89736e0d12"
Sep 14 07:55:26.093199 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:26.093183    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e98a7cbd0fbeb39f46f78cc6936f073c268de37d79a92f37a5c66a89736e0d12\": container with ID starting with e98a7cbd0fbeb39f46f78cc6936f073c268de37d79a92f37a5c66a89736e0d12 not found: ID does not exist" containerID="e98a7cbd0fbeb39f46f78cc6936f073c268de37d79a92f37a5c66a89736e0d12"
Sep 14 07:55:26.093293 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:26.093207    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98a7cbd0fbeb39f46f78cc6936f073c268de37d79a92f37a5c66a89736e0d12"} err="failed to get container status \"e98a7cbd0fbeb39f46f78cc6936f073c268de37d79a92f37a5c66a89736e0d12\": rpc error: code = NotFound desc = could not find container \"e98a7cbd0fbeb39f46f78cc6936f073c268de37d79a92f37a5c66a89736e0d12\": container with ID starting with e98a7cbd0fbeb39f46f78cc6936f073c268de37d79a92f37a5c66a89736e0d12 not found: ID does not exist"
Sep 14 07:55:26.093293 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:26.093222    2563 scope.go:117] "RemoveContainer" containerID="f5a135dca16728954775584e9589dfa39e37ab9e881b5be8aefcd6c70b2cd4fe"
Sep 14 07:55:26.101587 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:26.101572    2563 scope.go:117] "RemoveContainer" containerID="f5a135dca16728954775584e9589dfa39e37ab9e881b5be8aefcd6c70b2cd4fe"
Sep 14 07:55:26.101890 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:26.101871    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5a135dca16728954775584e9589dfa39e37ab9e881b5be8aefcd6c70b2cd4fe\": container with ID starting with f5a135dca16728954775584e9589dfa39e37ab9e881b5be8aefcd6c70b2cd4fe not found: ID does not exist" containerID="f5a135dca16728954775584e9589dfa39e37ab9e881b5be8aefcd6c70b2cd4fe"
Sep 14 07:55:26.101944 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:26.101896    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5a135dca16728954775584e9589dfa39e37ab9e881b5be8aefcd6c70b2cd4fe"} err="failed to get container status \"f5a135dca16728954775584e9589dfa39e37ab9e881b5be8aefcd6c70b2cd4fe\": rpc error: code = NotFound desc = could not find container \"f5a135dca16728954775584e9589dfa39e37ab9e881b5be8aefcd6c70b2cd4fe\": container with ID starting with f5a135dca16728954775584e9589dfa39e37ab9e881b5be8aefcd6c70b2cd4fe not found: ID does not exist"
Sep 14 07:55:26.198079 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:26.198055    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-hk89x"]
Sep 14 07:55:26.199721 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:55:26.199695    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43ebfe24_78eb_4c7d_a77c_4ea96d27269a.slice/crio-7c4e805f93506b1135b1b9e4534089de18d1c2becd012e0899357ed75bfa5597 WatchSource:0}: Error finding container 7c4e805f93506b1135b1b9e4534089de18d1c2becd012e0899357ed75bfa5597: Status 404 returned error can't find the container with id 7c4e805f93506b1135b1b9e4534089de18d1c2becd012e0899357ed75bfa5597
Sep 14 07:55:27.092250 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.092211    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk" event={"ID":"76d78b20-acce-4d41-bc23-6a17653a0a90","Type":"ContainerStarted","Data":"afbee21a40dccba890c9d79a179bac6352fd0533ac7705b05578463dc8b7e53b"}
Sep 14 07:55:27.092735 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.092368    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"
Sep 14 07:55:27.092735 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.092403    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk" event={"ID":"76d78b20-acce-4d41-bc23-6a17653a0a90","Type":"ContainerStarted","Data":"0b67964a87d76fcac3112bffd34f4eb81db81998978ff1f9315908c57f1774a9"}
Sep 14 07:55:27.094021 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.093989    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb" podUID="922842b7-6334-48e4-b6c4-c23fcce6593e" containerName="manager" containerID="cri-o://f80c15b6479aedcbbf545fae1522bdd9dfc73c1a61ef1d0b710a3cd1a9b3c2e3" gracePeriod=2
Sep 14 07:55:27.097749 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.097725    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hk89x" event={"ID":"43ebfe24-78eb-4c7d-a77c-4ea96d27269a","Type":"ContainerStarted","Data":"2e828185244fb9ae9843441de6d58506601d33ad2afcf75a50740a35f652201a"}
Sep 14 07:55:27.097832 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.097772    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hk89x" event={"ID":"43ebfe24-78eb-4c7d-a77c-4ea96d27269a","Type":"ContainerStarted","Data":"7c4e805f93506b1135b1b9e4534089de18d1c2becd012e0899357ed75bfa5597"}
Sep 14 07:55:27.097832 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.097807    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hk89x"
Sep 14 07:55:27.113934 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.113885    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk" podStartSLOduration=2.113867096 podStartE2EDuration="2.113867096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:27.112412627 +0000 UTC m=+447.741300278" watchObservedRunningTime="2026-09-14 07:55:27.113867096 +0000 UTC m=+447.742754725"
Sep 14 07:55:27.129740 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:27.129711    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-jhcsb\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="922842b7-6334-48e4-b6c4-c23fcce6593e" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:27.130158 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.130124    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hk89x" podStartSLOduration=2.130109736 podStartE2EDuration="2.130109736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:27.128202987 +0000 UTC m=+447.757090633" watchObservedRunningTime="2026-09-14 07:55:27.130109736 +0000 UTC m=+447.758997362"
Sep 14 07:55:27.330375 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.330349    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:27.332358 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:27.332333    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-jhcsb\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="922842b7-6334-48e4-b6c4-c23fcce6593e" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:27.393461 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.393385    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/922842b7-6334-48e4-b6c4-c23fcce6593e-extensions-socket-volume\") pod \"922842b7-6334-48e4-b6c4-c23fcce6593e\" (UID: \"922842b7-6334-48e4-b6c4-c23fcce6593e\") "
Sep 14 07:55:27.393461 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.393436    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlk2b\" (UniqueName: \"kubernetes.io/projected/922842b7-6334-48e4-b6c4-c23fcce6593e-kube-api-access-vlk2b\") pod \"922842b7-6334-48e4-b6c4-c23fcce6593e\" (UID: \"922842b7-6334-48e4-b6c4-c23fcce6593e\") "
Sep 14 07:55:27.393696 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.393669    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/922842b7-6334-48e4-b6c4-c23fcce6593e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "922842b7-6334-48e4-b6c4-c23fcce6593e" (UID: "922842b7-6334-48e4-b6c4-c23fcce6593e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:55:27.395441 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.395422    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/922842b7-6334-48e4-b6c4-c23fcce6593e-kube-api-access-vlk2b" (OuterVolumeSpecName: "kube-api-access-vlk2b") pod "922842b7-6334-48e4-b6c4-c23fcce6593e" (UID: "922842b7-6334-48e4-b6c4-c23fcce6593e"). InnerVolumeSpecName "kube-api-access-vlk2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:55:27.495090 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.495055    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/922842b7-6334-48e4-b6c4-c23fcce6593e-extensions-socket-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:55:27.495090 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.495084    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlk2b\" (UniqueName: \"kubernetes.io/projected/922842b7-6334-48e4-b6c4-c23fcce6593e-kube-api-access-vlk2b\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:55:27.885111 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:27.885078    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="922842b7-6334-48e4-b6c4-c23fcce6593e" path="/var/lib/kubelet/pods/922842b7-6334-48e4-b6c4-c23fcce6593e/volumes"
Sep 14 07:55:28.104167 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:28.104134    2563 generic.go:356] "Generic (PLEG): container finished" podID="922842b7-6334-48e4-b6c4-c23fcce6593e" containerID="f80c15b6479aedcbbf545fae1522bdd9dfc73c1a61ef1d0b710a3cd1a9b3c2e3" exitCode=2
Sep 14 07:55:28.104626 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:28.104182    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:28.104626 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:28.104235    2563 scope.go:117] "RemoveContainer" containerID="f80c15b6479aedcbbf545fae1522bdd9dfc73c1a61ef1d0b710a3cd1a9b3c2e3"
Sep 14 07:55:28.112991 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:28.112973    2563 scope.go:117] "RemoveContainer" containerID="f80c15b6479aedcbbf545fae1522bdd9dfc73c1a61ef1d0b710a3cd1a9b3c2e3"
Sep 14 07:55:28.113247 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:28.113229    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f80c15b6479aedcbbf545fae1522bdd9dfc73c1a61ef1d0b710a3cd1a9b3c2e3\": container with ID starting with f80c15b6479aedcbbf545fae1522bdd9dfc73c1a61ef1d0b710a3cd1a9b3c2e3 not found: ID does not exist" containerID="f80c15b6479aedcbbf545fae1522bdd9dfc73c1a61ef1d0b710a3cd1a9b3c2e3"
Sep 14 07:55:28.113292 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:28.113257    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f80c15b6479aedcbbf545fae1522bdd9dfc73c1a61ef1d0b710a3cd1a9b3c2e3"} err="failed to get container status \"f80c15b6479aedcbbf545fae1522bdd9dfc73c1a61ef1d0b710a3cd1a9b3c2e3\": rpc error: code = NotFound desc = could not find container \"f80c15b6479aedcbbf545fae1522bdd9dfc73c1a61ef1d0b710a3cd1a9b3c2e3\": container with ID starting with f80c15b6479aedcbbf545fae1522bdd9dfc73c1a61ef1d0b710a3cd1a9b3c2e3 not found: ID does not exist"
Sep 14 07:55:28.114377 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:28.114355    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-jhcsb\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="922842b7-6334-48e4-b6c4-c23fcce6593e" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:29.884441 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:29.884397    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-jhcsb\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="922842b7-6334-48e4-b6c4-c23fcce6593e" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-jhcsb"
Sep 14 07:55:38.107249 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:38.107218    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hk89x"
Sep 14 07:55:38.107735 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:38.107366    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"
Sep 14 07:55:38.382269 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:38.382191    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"]
Sep 14 07:55:38.391117 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:38.391097    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"
Sep 14 07:55:38.400274 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:38.400249    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"]
Sep 14 07:55:38.497566 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:38.497531    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkhds\" (UniqueName: \"kubernetes.io/projected/247a0b3c-d216-4ff2-9560-8dd722dc6412-kube-api-access-mkhds\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f65ml\" (UID: \"247a0b3c-d216-4ff2-9560-8dd722dc6412\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"
Sep 14 07:55:38.497566 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:38.497569    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/247a0b3c-d216-4ff2-9560-8dd722dc6412-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f65ml\" (UID: \"247a0b3c-d216-4ff2-9560-8dd722dc6412\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"
Sep 14 07:55:38.598695 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:38.598663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkhds\" (UniqueName: \"kubernetes.io/projected/247a0b3c-d216-4ff2-9560-8dd722dc6412-kube-api-access-mkhds\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f65ml\" (UID: \"247a0b3c-d216-4ff2-9560-8dd722dc6412\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"
Sep 14 07:55:38.598695 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:38.598699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/247a0b3c-d216-4ff2-9560-8dd722dc6412-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f65ml\" (UID: \"247a0b3c-d216-4ff2-9560-8dd722dc6412\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"
Sep 14 07:55:38.599115 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:38.599096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/247a0b3c-d216-4ff2-9560-8dd722dc6412-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f65ml\" (UID: \"247a0b3c-d216-4ff2-9560-8dd722dc6412\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"
Sep 14 07:55:38.611906 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:38.611876    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkhds\" (UniqueName: \"kubernetes.io/projected/247a0b3c-d216-4ff2-9560-8dd722dc6412-kube-api-access-mkhds\") pod \"kuadrant-operator-controller-manager-5769f99d9f-f65ml\" (UID: \"247a0b3c-d216-4ff2-9560-8dd722dc6412\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"
Sep 14 07:55:38.702449 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:38.702376    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"
Sep 14 07:55:38.976668 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:38.976611    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"]
Sep 14 07:55:38.978095 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:55:38.978072    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod247a0b3c_d216_4ff2_9560_8dd722dc6412.slice/crio-a8156a871019a21301f3f37142a0a3c750388ee543b80009889d09362e4d3fc2 WatchSource:0}: Error finding container a8156a871019a21301f3f37142a0a3c750388ee543b80009889d09362e4d3fc2: Status 404 returned error can't find the container with id a8156a871019a21301f3f37142a0a3c750388ee543b80009889d09362e4d3fc2
Sep 14 07:55:39.168494 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:39.168453    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml" event={"ID":"247a0b3c-d216-4ff2-9560-8dd722dc6412","Type":"ContainerStarted","Data":"41543f37d1b1a8ac29bbfa7b790ec77e71f17551e0e56220680c820e86813415"}
Sep 14 07:55:39.168979 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:39.168508    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml" event={"ID":"247a0b3c-d216-4ff2-9560-8dd722dc6412","Type":"ContainerStarted","Data":"a8156a871019a21301f3f37142a0a3c750388ee543b80009889d09362e4d3fc2"}
Sep 14 07:55:39.168979 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:39.168539    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"
Sep 14 07:55:39.203884 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:39.203834    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml" podStartSLOduration=1.203817144 podStartE2EDuration="1.203817144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:39.194559607 +0000 UTC m=+459.823447233" watchObservedRunningTime="2026-09-14 07:55:39.203817144 +0000 UTC m=+459.832704770"
Sep 14 07:55:50.176464 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:50.176433    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"
Sep 14 07:55:50.230157 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:50.230126    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"]
Sep 14 07:55:50.230341 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:50.230317    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk" podUID="76d78b20-acce-4d41-bc23-6a17653a0a90" containerName="manager" containerID="cri-o://afbee21a40dccba890c9d79a179bac6352fd0533ac7705b05578463dc8b7e53b" gracePeriod=10
Sep 14 07:55:50.472782 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:50.472759    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"
Sep 14 07:55:50.607773 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:50.607733    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76d78b20-acce-4d41-bc23-6a17653a0a90-extensions-socket-volume\") pod \"76d78b20-acce-4d41-bc23-6a17653a0a90\" (UID: \"76d78b20-acce-4d41-bc23-6a17653a0a90\") "
Sep 14 07:55:50.607938 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:50.607828    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2lgv\" (UniqueName: \"kubernetes.io/projected/76d78b20-acce-4d41-bc23-6a17653a0a90-kube-api-access-g2lgv\") pod \"76d78b20-acce-4d41-bc23-6a17653a0a90\" (UID: \"76d78b20-acce-4d41-bc23-6a17653a0a90\") "
Sep 14 07:55:50.608197 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:50.608170    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76d78b20-acce-4d41-bc23-6a17653a0a90-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "76d78b20-acce-4d41-bc23-6a17653a0a90" (UID: "76d78b20-acce-4d41-bc23-6a17653a0a90"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:55:50.609915 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:50.609890    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d78b20-acce-4d41-bc23-6a17653a0a90-kube-api-access-g2lgv" (OuterVolumeSpecName: "kube-api-access-g2lgv") pod "76d78b20-acce-4d41-bc23-6a17653a0a90" (UID: "76d78b20-acce-4d41-bc23-6a17653a0a90"). InnerVolumeSpecName "kube-api-access-g2lgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:55:50.708474 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:50.708409    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76d78b20-acce-4d41-bc23-6a17653a0a90-extensions-socket-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:55:50.708474 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:50.708431    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2lgv\" (UniqueName: \"kubernetes.io/projected/76d78b20-acce-4d41-bc23-6a17653a0a90-kube-api-access-g2lgv\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:55:51.235495 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:51.235435    2563 generic.go:356] "Generic (PLEG): container finished" podID="76d78b20-acce-4d41-bc23-6a17653a0a90" containerID="afbee21a40dccba890c9d79a179bac6352fd0533ac7705b05578463dc8b7e53b" exitCode=0
Sep 14 07:55:51.235495 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:51.235496    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"
Sep 14 07:55:51.236079 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:51.235525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk" event={"ID":"76d78b20-acce-4d41-bc23-6a17653a0a90","Type":"ContainerDied","Data":"afbee21a40dccba890c9d79a179bac6352fd0533ac7705b05578463dc8b7e53b"}
Sep 14 07:55:51.236079 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:51.235579    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk" event={"ID":"76d78b20-acce-4d41-bc23-6a17653a0a90","Type":"ContainerDied","Data":"0b67964a87d76fcac3112bffd34f4eb81db81998978ff1f9315908c57f1774a9"}
Sep 14 07:55:51.236079 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:51.235603    2563 scope.go:117] "RemoveContainer" containerID="afbee21a40dccba890c9d79a179bac6352fd0533ac7705b05578463dc8b7e53b"
Sep 14 07:55:51.244970 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:51.244961    2563 scope.go:117] "RemoveContainer" containerID="afbee21a40dccba890c9d79a179bac6352fd0533ac7705b05578463dc8b7e53b"
Sep 14 07:55:51.245215 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:55:51.245197    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afbee21a40dccba890c9d79a179bac6352fd0533ac7705b05578463dc8b7e53b\": container with ID starting with afbee21a40dccba890c9d79a179bac6352fd0533ac7705b05578463dc8b7e53b not found: ID does not exist" containerID="afbee21a40dccba890c9d79a179bac6352fd0533ac7705b05578463dc8b7e53b"
Sep 14 07:55:51.245261 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:51.245225    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afbee21a40dccba890c9d79a179bac6352fd0533ac7705b05578463dc8b7e53b"} err="failed to get container status \"afbee21a40dccba890c9d79a179bac6352fd0533ac7705b05578463dc8b7e53b\": rpc error: code = NotFound desc = could not find container \"afbee21a40dccba890c9d79a179bac6352fd0533ac7705b05578463dc8b7e53b\": container with ID starting with afbee21a40dccba890c9d79a179bac6352fd0533ac7705b05578463dc8b7e53b not found: ID does not exist"
Sep 14 07:55:51.259018 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:51.258995    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"]
Sep 14 07:55:51.265211 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:51.265191    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-hktdk"]
Sep 14 07:55:51.885239 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:55:51.885210    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76d78b20-acce-4d41-bc23-6a17653a0a90" path="/var/lib/kubelet/pods/76d78b20-acce-4d41-bc23-6a17653a0a90/volumes"
Sep 14 07:56:07.512063 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.512024    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-87jgn"]
Sep 14 07:56:07.512564 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.512514    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d78b20-acce-4d41-bc23-6a17653a0a90" containerName="manager"
Sep 14 07:56:07.512564 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.512533    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d78b20-acce-4d41-bc23-6a17653a0a90" containerName="manager"
Sep 14 07:56:07.512671 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.512604    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d78b20-acce-4d41-bc23-6a17653a0a90" containerName="manager"
Sep 14 07:56:07.515648 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.515622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-87jgn"
Sep 14 07:56:07.518070 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.518043    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 14 07:56:07.518195 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.518073    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-v4tmp\""
Sep 14 07:56:07.523743 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.523719    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-87jgn"]
Sep 14 07:56:07.536250 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.536218    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e6580abc-8bad-4240-aa3f-2c4f88d42949-config-file\") pod \"limitador-limitador-8588749977-87jgn\" (UID: \"e6580abc-8bad-4240-aa3f-2c4f88d42949\") " pod="kuadrant-system/limitador-limitador-8588749977-87jgn"
Sep 14 07:56:07.536416 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.536293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9wxv\" (UniqueName: \"kubernetes.io/projected/e6580abc-8bad-4240-aa3f-2c4f88d42949-kube-api-access-g9wxv\") pod \"limitador-limitador-8588749977-87jgn\" (UID: \"e6580abc-8bad-4240-aa3f-2c4f88d42949\") " pod="kuadrant-system/limitador-limitador-8588749977-87jgn"
Sep 14 07:56:07.612427 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.612388    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-87jgn"]
Sep 14 07:56:07.636714 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.636684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9wxv\" (UniqueName: \"kubernetes.io/projected/e6580abc-8bad-4240-aa3f-2c4f88d42949-kube-api-access-g9wxv\") pod \"limitador-limitador-8588749977-87jgn\" (UID: \"e6580abc-8bad-4240-aa3f-2c4f88d42949\") " pod="kuadrant-system/limitador-limitador-8588749977-87jgn"
Sep 14 07:56:07.636876 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.636744    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e6580abc-8bad-4240-aa3f-2c4f88d42949-config-file\") pod \"limitador-limitador-8588749977-87jgn\" (UID: \"e6580abc-8bad-4240-aa3f-2c4f88d42949\") " pod="kuadrant-system/limitador-limitador-8588749977-87jgn"
Sep 14 07:56:07.637337 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.637313    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e6580abc-8bad-4240-aa3f-2c4f88d42949-config-file\") pod \"limitador-limitador-8588749977-87jgn\" (UID: \"e6580abc-8bad-4240-aa3f-2c4f88d42949\") " pod="kuadrant-system/limitador-limitador-8588749977-87jgn"
Sep 14 07:56:07.645039 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.645016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9wxv\" (UniqueName: \"kubernetes.io/projected/e6580abc-8bad-4240-aa3f-2c4f88d42949-kube-api-access-g9wxv\") pod \"limitador-limitador-8588749977-87jgn\" (UID: \"e6580abc-8bad-4240-aa3f-2c4f88d42949\") " pod="kuadrant-system/limitador-limitador-8588749977-87jgn"
Sep 14 07:56:07.827146 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:07.826974    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-87jgn"
Sep 14 07:56:08.104051 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:08.104014    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-87jgn"]
Sep 14 07:56:08.329405 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:08.329354    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-87jgn" event={"ID":"e6580abc-8bad-4240-aa3f-2c4f88d42949","Type":"ContainerStarted","Data":"642f394d27eb43fa41c7856e09b5a97f42da85386f1fae29ee1f0fc6cbcd9907"}
Sep 14 07:56:10.340702 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:10.340664    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-87jgn" event={"ID":"e6580abc-8bad-4240-aa3f-2c4f88d42949","Type":"ContainerStarted","Data":"df834f4b5d34af24cb3f79d8f9c288a57c676720141e097065ed2e9536871f62"}
Sep 14 07:56:10.341072 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:10.340767    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-87jgn"
Sep 14 07:56:10.357191 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:10.357139    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-87jgn" podStartSLOduration=2.120327134 podStartE2EDuration="3.357122329s" totalImagesPullingTime="1.236795195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:56:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:56:10.356790127 +0000 UTC m=+490.985677781" watchObservedRunningTime="2026-09-14 07:56:10.357122329 +0000 UTC m=+490.986009956"
Sep 14 07:56:21.240799 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:21.240765    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-87jgn"]
Sep 14 07:56:21.241333 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:21.241017    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-87jgn" podUID="e6580abc-8bad-4240-aa3f-2c4f88d42949" containerName="limitador" containerID="cri-o://df834f4b5d34af24cb3f79d8f9c288a57c676720141e097065ed2e9536871f62" gracePeriod=30
Sep 14 07:56:21.241908 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:21.241753    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-87jgn"
Sep 14 07:56:21.774513 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:21.774486    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-87jgn"
Sep 14 07:56:21.855205 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:21.855175    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9wxv\" (UniqueName: \"kubernetes.io/projected/e6580abc-8bad-4240-aa3f-2c4f88d42949-kube-api-access-g9wxv\") pod \"e6580abc-8bad-4240-aa3f-2c4f88d42949\" (UID: \"e6580abc-8bad-4240-aa3f-2c4f88d42949\") "
Sep 14 07:56:21.855378 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:21.855260    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e6580abc-8bad-4240-aa3f-2c4f88d42949-config-file\") pod \"e6580abc-8bad-4240-aa3f-2c4f88d42949\" (UID: \"e6580abc-8bad-4240-aa3f-2c4f88d42949\") "
Sep 14 07:56:21.855628 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:21.855604    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6580abc-8bad-4240-aa3f-2c4f88d42949-config-file" (OuterVolumeSpecName: "config-file") pod "e6580abc-8bad-4240-aa3f-2c4f88d42949" (UID: "e6580abc-8bad-4240-aa3f-2c4f88d42949"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:56:21.857170 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:21.857150    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6580abc-8bad-4240-aa3f-2c4f88d42949-kube-api-access-g9wxv" (OuterVolumeSpecName: "kube-api-access-g9wxv") pod "e6580abc-8bad-4240-aa3f-2c4f88d42949" (UID: "e6580abc-8bad-4240-aa3f-2c4f88d42949"). InnerVolumeSpecName "kube-api-access-g9wxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:56:21.956352 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:21.956320    2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e6580abc-8bad-4240-aa3f-2c4f88d42949-config-file\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:56:21.956352 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:21.956345    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9wxv\" (UniqueName: \"kubernetes.io/projected/e6580abc-8bad-4240-aa3f-2c4f88d42949-kube-api-access-g9wxv\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:56:22.403986 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:22.403955    2563 generic.go:356] "Generic (PLEG): container finished" podID="e6580abc-8bad-4240-aa3f-2c4f88d42949" containerID="df834f4b5d34af24cb3f79d8f9c288a57c676720141e097065ed2e9536871f62" exitCode=0
Sep 14 07:56:22.404461 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:22.404001    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-87jgn"
Sep 14 07:56:22.404461 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:22.404042    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-87jgn" event={"ID":"e6580abc-8bad-4240-aa3f-2c4f88d42949","Type":"ContainerDied","Data":"df834f4b5d34af24cb3f79d8f9c288a57c676720141e097065ed2e9536871f62"}
Sep 14 07:56:22.404461 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:22.404095    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-87jgn" event={"ID":"e6580abc-8bad-4240-aa3f-2c4f88d42949","Type":"ContainerDied","Data":"642f394d27eb43fa41c7856e09b5a97f42da85386f1fae29ee1f0fc6cbcd9907"}
Sep 14 07:56:22.404461 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:22.404120    2563 scope.go:117] "RemoveContainer" containerID="df834f4b5d34af24cb3f79d8f9c288a57c676720141e097065ed2e9536871f62"
Sep 14 07:56:22.412868 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:22.412851    2563 scope.go:117] "RemoveContainer" containerID="df834f4b5d34af24cb3f79d8f9c288a57c676720141e097065ed2e9536871f62"
Sep 14 07:56:22.413098 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:56:22.413075    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df834f4b5d34af24cb3f79d8f9c288a57c676720141e097065ed2e9536871f62\": container with ID starting with df834f4b5d34af24cb3f79d8f9c288a57c676720141e097065ed2e9536871f62 not found: ID does not exist" containerID="df834f4b5d34af24cb3f79d8f9c288a57c676720141e097065ed2e9536871f62"
Sep 14 07:56:22.413166 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:22.413111    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df834f4b5d34af24cb3f79d8f9c288a57c676720141e097065ed2e9536871f62"} err="failed to get container status \"df834f4b5d34af24cb3f79d8f9c288a57c676720141e097065ed2e9536871f62\": rpc error: code = NotFound desc = could not find container \"df834f4b5d34af24cb3f79d8f9c288a57c676720141e097065ed2e9536871f62\": container with ID starting with df834f4b5d34af24cb3f79d8f9c288a57c676720141e097065ed2e9536871f62 not found: ID does not exist"
Sep 14 07:56:22.429845 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:22.429820    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-87jgn"]
Sep 14 07:56:22.435496 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:22.435473    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-87jgn"]
Sep 14 07:56:23.886150 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:23.886120    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6580abc-8bad-4240-aa3f-2c4f88d42949" path="/var/lib/kubelet/pods/e6580abc-8bad-4240-aa3f-2c4f88d42949/volumes"
Sep 14 07:56:24.048806 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.048776    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw"]
Sep 14 07:56:24.049117 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.049106    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6580abc-8bad-4240-aa3f-2c4f88d42949" containerName="limitador"
Sep 14 07:56:24.049171 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.049119    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6580abc-8bad-4240-aa3f-2c4f88d42949" containerName="limitador"
Sep 14 07:56:24.049206 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.049171    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6580abc-8bad-4240-aa3f-2c4f88d42949" containerName="limitador"
Sep 14 07:56:24.052202 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.052186    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw"
Sep 14 07:56:24.055431 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.055412    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-ph7db\""
Sep 14 07:56:24.055567 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.055449    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 14 07:56:24.056012 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.055999    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 07:56:24.060921 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.060905    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 07:56:24.070935 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.070914    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw"]
Sep 14 07:56:24.175778 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.175704    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgzgt\" (UniqueName: \"kubernetes.io/projected/02642698-3e24-4da5-9b12-6aa77b31f925-kube-api-access-fgzgt\") pod \"postgres-69fb4594cf-bkpkw\" (UID: \"02642698-3e24-4da5-9b12-6aa77b31f925\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw"
Sep 14 07:56:24.175778 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.175762    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/02642698-3e24-4da5-9b12-6aa77b31f925-data\") pod \"postgres-69fb4594cf-bkpkw\" (UID: \"02642698-3e24-4da5-9b12-6aa77b31f925\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw"
Sep 14 07:56:24.276207 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.276174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgzgt\" (UniqueName: \"kubernetes.io/projected/02642698-3e24-4da5-9b12-6aa77b31f925-kube-api-access-fgzgt\") pod \"postgres-69fb4594cf-bkpkw\" (UID: \"02642698-3e24-4da5-9b12-6aa77b31f925\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw"
Sep 14 07:56:24.276362 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.276241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/02642698-3e24-4da5-9b12-6aa77b31f925-data\") pod \"postgres-69fb4594cf-bkpkw\" (UID: \"02642698-3e24-4da5-9b12-6aa77b31f925\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw"
Sep 14 07:56:24.276604 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.276586    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/02642698-3e24-4da5-9b12-6aa77b31f925-data\") pod \"postgres-69fb4594cf-bkpkw\" (UID: \"02642698-3e24-4da5-9b12-6aa77b31f925\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw"
Sep 14 07:56:24.286034 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.286009    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgzgt\" (UniqueName: \"kubernetes.io/projected/02642698-3e24-4da5-9b12-6aa77b31f925-kube-api-access-fgzgt\") pod \"postgres-69fb4594cf-bkpkw\" (UID: \"02642698-3e24-4da5-9b12-6aa77b31f925\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw"
Sep 14 07:56:24.362252 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.362221    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw"
Sep 14 07:56:24.639426 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:24.639401    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw"]
Sep 14 07:56:24.641251 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:56:24.641210    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02642698_3e24_4da5_9b12_6aa77b31f925.slice/crio-ebac4f9527ff17c4b95330906430ed6c79ba14015c69e07ba46744a479c8a66f WatchSource:0}: Error finding container ebac4f9527ff17c4b95330906430ed6c79ba14015c69e07ba46744a479c8a66f: Status 404 returned error can't find the container with id ebac4f9527ff17c4b95330906430ed6c79ba14015c69e07ba46744a479c8a66f
Sep 14 07:56:25.430506 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:25.430464    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw" event={"ID":"02642698-3e24-4da5-9b12-6aa77b31f925","Type":"ContainerStarted","Data":"ebac4f9527ff17c4b95330906430ed6c79ba14015c69e07ba46744a479c8a66f"}
Sep 14 07:56:29.457091 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:29.457053    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw" event={"ID":"02642698-3e24-4da5-9b12-6aa77b31f925","Type":"ContainerStarted","Data":"c5a4456872c39cb0897a0e1e420ccca458b1887e83731d529fb76719752320ab"}
Sep 14 07:56:29.457534 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:29.457227    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw"
Sep 14 07:56:29.473204 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:29.473147    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw" podStartSLOduration=0.75995239 podStartE2EDuration="5.473127753s" totalImagesPullingTime="4.713175363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:56:29.472997025 +0000 UTC m=+510.101884663" watchObservedRunningTime="2026-09-14 07:56:29.473127753 +0000 UTC m=+510.102015380"
Sep 14 07:56:35.492343 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:35.492314    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bkpkw"
Sep 14 07:56:43.826798 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:43.826763    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d47cd7dd-xctvf"]
Sep 14 07:56:43.854453 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:43.854411    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d47cd7dd-xctvf"]
Sep 14 07:56:43.854634 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:43.854509    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:43.857004 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:43.856973    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 07:56:43.857170 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:43.857144    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5j4xp\""
Sep 14 07:56:43.857256 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:43.857188    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 07:56:43.857736 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:43.857716    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 07:56:43.948205 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:43.948174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-metrics-certs\") pod \"maas-controller-6d47cd7dd-xctvf\" (UID: \"b32aa893-cce8-4ba0-aba2-668e935d448f\") " pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:43.948376 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:43.948214    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-webhook-cert\") pod \"maas-controller-6d47cd7dd-xctvf\" (UID: \"b32aa893-cce8-4ba0-aba2-668e935d448f\") " pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:43.948376 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:43.948304    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22kkl\" (UniqueName: \"kubernetes.io/projected/b32aa893-cce8-4ba0-aba2-668e935d448f-kube-api-access-22kkl\") pod \"maas-controller-6d47cd7dd-xctvf\" (UID: \"b32aa893-cce8-4ba0-aba2-668e935d448f\") " pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:44.024233 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.024199    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6945fd44b-qgvbb"]
Sep 14 07:56:44.027941 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.027920    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 07:56:44.038875 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.038846    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6945fd44b-qgvbb"]
Sep 14 07:56:44.049333 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.049306    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-metrics-certs\") pod \"maas-controller-6d47cd7dd-xctvf\" (UID: \"b32aa893-cce8-4ba0-aba2-668e935d448f\") " pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:44.049597 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.049576    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-webhook-cert\") pod \"maas-controller-6d47cd7dd-xctvf\" (UID: \"b32aa893-cce8-4ba0-aba2-668e935d448f\") " pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:44.049862 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:56:44.049474    2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 14 07:56:44.049948 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.049859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22kkl\" (UniqueName: \"kubernetes.io/projected/b32aa893-cce8-4ba0-aba2-668e935d448f-kube-api-access-22kkl\") pod \"maas-controller-6d47cd7dd-xctvf\" (UID: \"b32aa893-cce8-4ba0-aba2-668e935d448f\") " pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:44.049948 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:56:44.049915    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-metrics-certs podName:b32aa893-cce8-4ba0-aba2-668e935d448f nodeName:}" failed. No retries permitted until 2026-09-14 07:56:44.549892596 +0000 UTC m=+525.178780217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-metrics-certs") pod "maas-controller-6d47cd7dd-xctvf" (UID: "b32aa893-cce8-4ba0-aba2-668e935d448f") : secret "maas-controller-metrics-tls" not found
Sep 14 07:56:44.052401 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.052378    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-webhook-cert\") pod \"maas-controller-6d47cd7dd-xctvf\" (UID: \"b32aa893-cce8-4ba0-aba2-668e935d448f\") " pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:44.061230 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.061211    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22kkl\" (UniqueName: \"kubernetes.io/projected/b32aa893-cce8-4ba0-aba2-668e935d448f-kube-api-access-22kkl\") pod \"maas-controller-6d47cd7dd-xctvf\" (UID: \"b32aa893-cce8-4ba0-aba2-668e935d448f\") " pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:44.150991 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.150918    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-metrics-certs\") pod \"maas-controller-6945fd44b-qgvbb\" (UID: \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\") " pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 07:56:44.150991 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.150959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-webhook-cert\") pod \"maas-controller-6945fd44b-qgvbb\" (UID: \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\") " pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 07:56:44.151129 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.151032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct7d9\" (UniqueName: \"kubernetes.io/projected/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-kube-api-access-ct7d9\") pod \"maas-controller-6945fd44b-qgvbb\" (UID: \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\") " pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 07:56:44.252313 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.252278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct7d9\" (UniqueName: \"kubernetes.io/projected/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-kube-api-access-ct7d9\") pod \"maas-controller-6945fd44b-qgvbb\" (UID: \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\") " pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 07:56:44.252500 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.252341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-metrics-certs\") pod \"maas-controller-6945fd44b-qgvbb\" (UID: \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\") " pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 07:56:44.252500 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.252370    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-webhook-cert\") pod \"maas-controller-6945fd44b-qgvbb\" (UID: \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\") " pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 07:56:44.254852 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.254828    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-metrics-certs\") pod \"maas-controller-6945fd44b-qgvbb\" (UID: \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\") " pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 07:56:44.254953 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.254853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-webhook-cert\") pod \"maas-controller-6945fd44b-qgvbb\" (UID: \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\") " pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 07:56:44.261461 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.261436    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct7d9\" (UniqueName: \"kubernetes.io/projected/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-kube-api-access-ct7d9\") pod \"maas-controller-6945fd44b-qgvbb\" (UID: \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\") " pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 07:56:44.338842 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.338813    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 07:56:44.555319 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.555237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-metrics-certs\") pod \"maas-controller-6d47cd7dd-xctvf\" (UID: \"b32aa893-cce8-4ba0-aba2-668e935d448f\") " pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:44.557677 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.557634    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-metrics-certs\") pod \"maas-controller-6d47cd7dd-xctvf\" (UID: \"b32aa893-cce8-4ba0-aba2-668e935d448f\") " pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:44.611068 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.611041    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6945fd44b-qgvbb"]
Sep 14 07:56:44.612177 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:56:44.612153    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f1dab9_82b8_49aa_8ea8_1f9c4abb88be.slice/crio-7c97295e7cdcbe099160f552775fbe7aa3539e42d9531b1b88288768c51b6553 WatchSource:0}: Error finding container 7c97295e7cdcbe099160f552775fbe7aa3539e42d9531b1b88288768c51b6553: Status 404 returned error can't find the container with id 7c97295e7cdcbe099160f552775fbe7aa3539e42d9531b1b88288768c51b6553
Sep 14 07:56:44.765681 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:44.765619    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:45.048710 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:45.048544    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d47cd7dd-xctvf"]
Sep 14 07:56:45.051343 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:56:45.051313    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32aa893_cce8_4ba0_aba2_668e935d448f.slice/crio-3da37baa0ec3c4b037719493956208829194f785dbc165502e631f6e2ed58e9f WatchSource:0}: Error finding container 3da37baa0ec3c4b037719493956208829194f785dbc165502e631f6e2ed58e9f: Status 404 returned error can't find the container with id 3da37baa0ec3c4b037719493956208829194f785dbc165502e631f6e2ed58e9f
Sep 14 07:56:45.544518 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:45.544482    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6945fd44b-qgvbb" event={"ID":"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be","Type":"ContainerStarted","Data":"7c97295e7cdcbe099160f552775fbe7aa3539e42d9531b1b88288768c51b6553"}
Sep 14 07:56:45.545699 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:45.545673    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d47cd7dd-xctvf" event={"ID":"b32aa893-cce8-4ba0-aba2-668e935d448f","Type":"ContainerStarted","Data":"3da37baa0ec3c4b037719493956208829194f785dbc165502e631f6e2ed58e9f"}
Sep 14 07:56:48.566820 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:48.566780    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6945fd44b-qgvbb" event={"ID":"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be","Type":"ContainerStarted","Data":"62c567877446d4bc7a6dcb73d21241bf918ec2a68b732c42f042fd83e07ae5de"}
Sep 14 07:56:48.567295 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:48.566961    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 07:56:48.568602 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:48.568572    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d47cd7dd-xctvf" event={"ID":"b32aa893-cce8-4ba0-aba2-668e935d448f","Type":"ContainerStarted","Data":"153f58ceda8111e9d3772f4251714fa88463358e734f16b3a44494f82e764fb5"}
Sep 14 07:56:48.568826 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:48.568682    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:48.587527 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:48.587473    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6945fd44b-qgvbb" podStartSLOduration=2.538941225 podStartE2EDuration="5.587455209s" totalImagesPullingTime="3.048513984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:56:48.587430612 +0000 UTC m=+529.216318269" watchObservedRunningTime="2026-09-14 07:56:48.587455209 +0000 UTC m=+529.216342834"
Sep 14 07:56:48.613833 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:48.613785    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d47cd7dd-xctvf" podStartSLOduration=2.997090784 podStartE2EDuration="5.613770181s" totalImagesPullingTime="2.616679397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:56:48.610487008 +0000 UTC m=+529.239374669" watchObservedRunningTime="2026-09-14 07:56:48.613770181 +0000 UTC m=+529.242657807"
Sep 14 07:56:59.576833 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:59.576751    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:59.576833 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:59.576826    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 07:56:59.642597 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:59.642556    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d47cd7dd-xctvf"]
Sep 14 07:56:59.642854 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:59.642824    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6d47cd7dd-xctvf" podUID="b32aa893-cce8-4ba0-aba2-668e935d448f" containerName="manager" containerID="cri-o://153f58ceda8111e9d3772f4251714fa88463358e734f16b3a44494f82e764fb5" gracePeriod=30
Sep 14 07:56:59.887950 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:59.887702    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:56:59.986908 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:59.986866    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-webhook-cert\") pod \"b32aa893-cce8-4ba0-aba2-668e935d448f\" (UID: \"b32aa893-cce8-4ba0-aba2-668e935d448f\") "
Sep 14 07:56:59.987101 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:59.986931    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-metrics-certs\") pod \"b32aa893-cce8-4ba0-aba2-668e935d448f\" (UID: \"b32aa893-cce8-4ba0-aba2-668e935d448f\") "
Sep 14 07:56:59.987101 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:59.986959    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22kkl\" (UniqueName: \"kubernetes.io/projected/b32aa893-cce8-4ba0-aba2-668e935d448f-kube-api-access-22kkl\") pod \"b32aa893-cce8-4ba0-aba2-668e935d448f\" (UID: \"b32aa893-cce8-4ba0-aba2-668e935d448f\") "
Sep 14 07:56:59.989126 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:59.989096    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b32aa893-cce8-4ba0-aba2-668e935d448f" (UID: "b32aa893-cce8-4ba0-aba2-668e935d448f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:56:59.989230 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:59.989135    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b32aa893-cce8-4ba0-aba2-668e935d448f" (UID: "b32aa893-cce8-4ba0-aba2-668e935d448f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:56:59.989230 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:56:59.989179    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b32aa893-cce8-4ba0-aba2-668e935d448f-kube-api-access-22kkl" (OuterVolumeSpecName: "kube-api-access-22kkl") pod "b32aa893-cce8-4ba0-aba2-668e935d448f" (UID: "b32aa893-cce8-4ba0-aba2-668e935d448f"). InnerVolumeSpecName "kube-api-access-22kkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:57:00.087823 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:00.087741    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-webhook-cert\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:57:00.087823 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:00.087777    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b32aa893-cce8-4ba0-aba2-668e935d448f-metrics-certs\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:57:00.087823 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:00.087787    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22kkl\" (UniqueName: \"kubernetes.io/projected/b32aa893-cce8-4ba0-aba2-668e935d448f-kube-api-access-22kkl\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 07:57:00.635367 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:00.635329    2563 generic.go:356] "Generic (PLEG): container finished" podID="b32aa893-cce8-4ba0-aba2-668e935d448f" containerID="153f58ceda8111e9d3772f4251714fa88463358e734f16b3a44494f82e764fb5" exitCode=0
Sep 14 07:57:00.635367 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:00.635372    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d47cd7dd-xctvf"
Sep 14 07:57:00.635839 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:00.635412    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d47cd7dd-xctvf" event={"ID":"b32aa893-cce8-4ba0-aba2-668e935d448f","Type":"ContainerDied","Data":"153f58ceda8111e9d3772f4251714fa88463358e734f16b3a44494f82e764fb5"}
Sep 14 07:57:00.635839 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:00.635465    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d47cd7dd-xctvf" event={"ID":"b32aa893-cce8-4ba0-aba2-668e935d448f","Type":"ContainerDied","Data":"3da37baa0ec3c4b037719493956208829194f785dbc165502e631f6e2ed58e9f"}
Sep 14 07:57:00.635839 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:00.635481    2563 scope.go:117] "RemoveContainer" containerID="153f58ceda8111e9d3772f4251714fa88463358e734f16b3a44494f82e764fb5"
Sep 14 07:57:00.645318 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:00.645178    2563 scope.go:117] "RemoveContainer" containerID="153f58ceda8111e9d3772f4251714fa88463358e734f16b3a44494f82e764fb5"
Sep 14 07:57:00.645507 ip-10-0-143-250 kubenswrapper[2563]: E0914 07:57:00.645480    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"153f58ceda8111e9d3772f4251714fa88463358e734f16b3a44494f82e764fb5\": container with ID starting with 153f58ceda8111e9d3772f4251714fa88463358e734f16b3a44494f82e764fb5 not found: ID does not exist" containerID="153f58ceda8111e9d3772f4251714fa88463358e734f16b3a44494f82e764fb5"
Sep 14 07:57:00.645559 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:00.645515    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"153f58ceda8111e9d3772f4251714fa88463358e734f16b3a44494f82e764fb5"} err="failed to get container status \"153f58ceda8111e9d3772f4251714fa88463358e734f16b3a44494f82e764fb5\": rpc error: code = NotFound desc = could not find container \"153f58ceda8111e9d3772f4251714fa88463358e734f16b3a44494f82e764fb5\": container with ID starting with 153f58ceda8111e9d3772f4251714fa88463358e734f16b3a44494f82e764fb5 not found: ID does not exist"
Sep 14 07:57:00.656842 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:00.656819    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d47cd7dd-xctvf"]
Sep 14 07:57:00.664191 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:00.664165    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d47cd7dd-xctvf"]
Sep 14 07:57:01.886027 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:01.885991    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b32aa893-cce8-4ba0-aba2-668e935d448f" path="/var/lib/kubelet/pods/b32aa893-cce8-4ba0-aba2-668e935d448f/volumes"
Sep 14 07:57:21.748303 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.748271    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-jhhdc"]
Sep 14 07:57:21.748897 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.748834    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b32aa893-cce8-4ba0-aba2-668e935d448f" containerName="manager"
Sep 14 07:57:21.748897 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.748853    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b32aa893-cce8-4ba0-aba2-668e935d448f" containerName="manager"
Sep 14 07:57:21.749021 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.748963    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b32aa893-cce8-4ba0-aba2-668e935d448f" containerName="manager"
Sep 14 07:57:21.761605 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.761571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-jhhdc"
Sep 14 07:57:21.764903 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.764880    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 14 07:57:21.765603 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.765018    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ld9b8\""
Sep 14 07:57:21.765732 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.765601    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-jhhdc"]
Sep 14 07:57:21.877976 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.877943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78828de1-4078-4002-a86b-da44a9b7120f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-jhhdc\" (UID: \"78828de1-4078-4002-a86b-da44a9b7120f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jhhdc"
Sep 14 07:57:21.878152 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.877991    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvfp4\" (UniqueName: \"kubernetes.io/projected/78828de1-4078-4002-a86b-da44a9b7120f-kube-api-access-jvfp4\") pod \"payload-processing-5b54b7fbb7-jhhdc\" (UID: \"78828de1-4078-4002-a86b-da44a9b7120f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jhhdc"
Sep 14 07:57:21.979100 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.979066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78828de1-4078-4002-a86b-da44a9b7120f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-jhhdc\" (UID: \"78828de1-4078-4002-a86b-da44a9b7120f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jhhdc"
Sep 14 07:57:21.979279 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.979111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvfp4\" (UniqueName: \"kubernetes.io/projected/78828de1-4078-4002-a86b-da44a9b7120f-kube-api-access-jvfp4\") pod \"payload-processing-5b54b7fbb7-jhhdc\" (UID: \"78828de1-4078-4002-a86b-da44a9b7120f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jhhdc"
Sep 14 07:57:21.979795 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.979772    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78828de1-4078-4002-a86b-da44a9b7120f-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-jhhdc\" (UID: \"78828de1-4078-4002-a86b-da44a9b7120f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jhhdc"
Sep 14 07:57:21.988126 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:21.988096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvfp4\" (UniqueName: \"kubernetes.io/projected/78828de1-4078-4002-a86b-da44a9b7120f-kube-api-access-jvfp4\") pod \"payload-processing-5b54b7fbb7-jhhdc\" (UID: \"78828de1-4078-4002-a86b-da44a9b7120f\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-jhhdc"
Sep 14 07:57:22.076688 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:22.076581    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-jhhdc"
Sep 14 07:57:22.372077 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:22.372009    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-jhhdc"]
Sep 14 07:57:22.374880 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:57:22.374851    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78828de1_4078_4002_a86b_da44a9b7120f.slice/crio-6d5982e8a3c0a30147512eefd2aa82434f4a0c52a0f0d886a818d11af7bda7f7 WatchSource:0}: Error finding container 6d5982e8a3c0a30147512eefd2aa82434f4a0c52a0f0d886a818d11af7bda7f7: Status 404 returned error can't find the container with id 6d5982e8a3c0a30147512eefd2aa82434f4a0c52a0f0d886a818d11af7bda7f7
Sep 14 07:57:22.760254 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:22.760164    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-jhhdc" event={"ID":"78828de1-4078-4002-a86b-da44a9b7120f","Type":"ContainerStarted","Data":"6d5982e8a3c0a30147512eefd2aa82434f4a0c52a0f0d886a818d11af7bda7f7"}
Sep 14 07:57:24.772518 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:24.772484    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-jhhdc" event={"ID":"78828de1-4078-4002-a86b-da44a9b7120f","Type":"ContainerStarted","Data":"4d6809d49304a9e41527ce86a3af83a397ecb1574835aad34ffdf1584ec833fa"}
Sep 14 07:57:24.772927 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:24.772550    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-jhhdc"
Sep 14 07:57:24.792512 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:24.792463    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-jhhdc" podStartSLOduration=2.435342991 podStartE2EDuration="3.792448735s" totalImagesPullingTime="1.357105744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:24.791181433 +0000 UTC m=+565.420069080" watchObservedRunningTime="2026-09-14 07:57:24.792448735 +0000 UTC m=+565.421336361"
Sep 14 07:57:35.780502 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:35.780465    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-jhhdc"
Sep 14 07:57:42.508578 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.508540    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"]
Sep 14 07:57:42.512591 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.512569    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.514923 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.514893    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 14 07:57:42.515465 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.515443    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 14 07:57:42.515568 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.515447    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7j9jg\""
Sep 14 07:57:42.515568 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.515454    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 14 07:57:42.521877 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.521849    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"]
Sep 14 07:57:42.560239 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.560210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6eb6f09f-01ee-44a0-961b-e288d02c820c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.560385 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.560255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6eb6f09f-01ee-44a0-961b-e288d02c820c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.560385 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.560343    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h6lh\" (UniqueName: \"kubernetes.io/projected/6eb6f09f-01ee-44a0-961b-e288d02c820c-kube-api-access-6h6lh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.560385 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.560374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6eb6f09f-01ee-44a0-961b-e288d02c820c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.560494 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.560398    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6eb6f09f-01ee-44a0-961b-e288d02c820c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.560494 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.560413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6eb6f09f-01ee-44a0-961b-e288d02c820c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.661597 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.661561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h6lh\" (UniqueName: \"kubernetes.io/projected/6eb6f09f-01ee-44a0-961b-e288d02c820c-kube-api-access-6h6lh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.661597 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.661600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6eb6f09f-01ee-44a0-961b-e288d02c820c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.661845 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.661621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6eb6f09f-01ee-44a0-961b-e288d02c820c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.661845 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.661654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6eb6f09f-01ee-44a0-961b-e288d02c820c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.661845 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.661728    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6eb6f09f-01ee-44a0-961b-e288d02c820c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.661845 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.661781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6eb6f09f-01ee-44a0-961b-e288d02c820c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.662042 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.662016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6eb6f09f-01ee-44a0-961b-e288d02c820c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.662097 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.662067    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6eb6f09f-01ee-44a0-961b-e288d02c820c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.662152 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.662092    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6eb6f09f-01ee-44a0-961b-e288d02c820c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.664159 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.664130    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6eb6f09f-01ee-44a0-961b-e288d02c820c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.664446 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.664427    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6eb6f09f-01ee-44a0-961b-e288d02c820c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.677842 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.677818    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h6lh\" (UniqueName: \"kubernetes.io/projected/6eb6f09f-01ee-44a0-961b-e288d02c820c-kube-api-access-6h6lh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65\" (UID: \"6eb6f09f-01ee-44a0-961b-e288d02c820c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:42.825189 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:42.825108    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:43.102971 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:43.102944    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"]
Sep 14 07:57:43.105020 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:57:43.104989    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eb6f09f_01ee_44a0_961b_e288d02c820c.slice/crio-222f0022e6af61b01ff607f24cc89f9874906c0b0c59ecf077f5a03e4c3cfe2c WatchSource:0}: Error finding container 222f0022e6af61b01ff607f24cc89f9874906c0b0c59ecf077f5a03e4c3cfe2c: Status 404 returned error can't find the container with id 222f0022e6af61b01ff607f24cc89f9874906c0b0c59ecf077f5a03e4c3cfe2c
Sep 14 07:57:43.886366 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:43.886331    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65" event={"ID":"6eb6f09f-01ee-44a0-961b-e288d02c820c","Type":"ContainerStarted","Data":"222f0022e6af61b01ff607f24cc89f9874906c0b0c59ecf077f5a03e4c3cfe2c"}
Sep 14 07:57:44.886034 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:44.885999    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65" event={"ID":"6eb6f09f-01ee-44a0-961b-e288d02c820c","Type":"ContainerStarted","Data":"f22de4e63e6d372623b6087230aad3cc2f1411f9b3ab650364164565278df8ee"}
Sep 14 07:57:44.904254 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:44.904200    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65" podStartSLOduration=1.8964292249999999 podStartE2EDuration="2.904182432s" totalImagesPullingTime="1.007753207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:44.902019831 +0000 UTC m=+585.530907479" watchObservedRunningTime="2026-09-14 07:57:44.904182432 +0000 UTC m=+585.533070110"
Sep 14 07:57:52.825774 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:52.825739    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:52.826235 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:52.826039    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:52.838758 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:52.838734    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:52.940896 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:52.940869    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65"
Sep 14 07:57:53.903911 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:53.903864    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"]
Sep 14 07:57:53.907592 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:53.907568    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:53.910690 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:53.910668    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 14 07:57:53.921452 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:53.921424    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"]
Sep 14 07:57:53.967792 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:53.967760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr7vz\" (UniqueName: \"kubernetes.io/projected/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-kube-api-access-pr7vz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:53.967956 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:53.967867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:53.968042 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:53.967961    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:53.968042 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:53.967983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:53.968042 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:53.968008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:53.968042 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:53.968024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.068868 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.068830    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.069055 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.068911    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.069055 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.068941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.069055 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.068976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.069223 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.069110    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.069223 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.069154    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr7vz\" (UniqueName: \"kubernetes.io/projected/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-kube-api-access-pr7vz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.069367 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.069347    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.069433 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.069397    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.069500 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.069424    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.071240 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.071216    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.071520 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.071500    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.079152 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.079135    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr7vz\" (UniqueName: \"kubernetes.io/projected/1aee68db-f5a3-48e1-b25e-d5ff2cf38833-kube-api-access-pr7vz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vbftw\" (UID: \"1aee68db-f5a3-48e1-b25e-d5ff2cf38833\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.220262 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.220173    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:57:54.506075 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.505987    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"]
Sep 14 07:57:54.509666 ip-10-0-143-250 kubenswrapper[2563]: W0914 07:57:54.509616    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aee68db_f5a3_48e1_b25e_d5ff2cf38833.slice/crio-512a5405d0c058fc46da81a4f63829111966ddba1fb2d9916352e02cad11e763 WatchSource:0}: Error finding container 512a5405d0c058fc46da81a4f63829111966ddba1fb2d9916352e02cad11e763: Status 404 returned error can't find the container with id 512a5405d0c058fc46da81a4f63829111966ddba1fb2d9916352e02cad11e763
Sep 14 07:57:54.944883 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.944845    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw" event={"ID":"1aee68db-f5a3-48e1-b25e-d5ff2cf38833","Type":"ContainerStarted","Data":"ffff7e014dedd71ea3a117848f7671c7119f144853b06c7e4d60ddb422beba51"}
Sep 14 07:57:54.945300 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.944915    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw" event={"ID":"1aee68db-f5a3-48e1-b25e-d5ff2cf38833","Type":"ContainerStarted","Data":"512a5405d0c058fc46da81a4f63829111966ddba1fb2d9916352e02cad11e763"}
Sep 14 07:57:54.967905 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:54.967855    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw" podStartSLOduration=1.7885377230000001 podStartE2EDuration="1.967836927s" totalImagesPullingTime="179.299204ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:54.965356586 +0000 UTC m=+595.594244212" watchObservedRunningTime="2026-09-14 07:57:54.967836927 +0000 UTC m=+595.596724609"
Sep 14 07:57:59.798454 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:59.798423    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 07:57:59.798864 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:57:59.798850    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 07:58:04.220562 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:58:04.220521    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:58:04.223241 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:58:04.220599    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:58:04.233593 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:58:04.233569    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 07:58:05.008115 ip-10-0-143-250 kubenswrapper[2563]: I0914 07:58:05.008088    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vbftw"
Sep 14 08:00:00.149934 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:00.149858    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822880-hllm7"]
Sep 14 08:00:00.153221 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:00.153203    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-hllm7"
Sep 14 08:00:00.155776 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:00.155757    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:00:00.161891 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:00.161865    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822880-hllm7"]
Sep 14 08:00:00.165364 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:00.165342    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-hllm7"
Sep 14 08:00:00.441712 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:00.441685    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822880-hllm7"]
Sep 14 08:00:00.443168 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:00:00.443130    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68393035_a1a6_462e_b4a1_935a74ac0d71.slice/crio-59f341e9b0f00e28ec31c83cf38f09542037b349ca4a2ef9d9b05e44e581d03c WatchSource:0}: Error finding container 59f341e9b0f00e28ec31c83cf38f09542037b349ca4a2ef9d9b05e44e581d03c: Status 404 returned error can't find the container with id 59f341e9b0f00e28ec31c83cf38f09542037b349ca4a2ef9d9b05e44e581d03c
Sep 14 08:00:00.445472 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:00.445456    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:00:00.642960 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:00.642921    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-hllm7" event={"ID":"68393035-a1a6-462e-b4a1-935a74ac0d71","Type":"ContainerStarted","Data":"59f341e9b0f00e28ec31c83cf38f09542037b349ca4a2ef9d9b05e44e581d03c"}
Sep 14 08:00:03.669217 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:03.669178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-hllm7" event={"ID":"68393035-a1a6-462e-b4a1-935a74ac0d71","Type":"ContainerStarted","Data":"5cf94aa4e00fe1abb0d76fd573ccaa435baba611678a1e1bfe5187449454c4a7"}
Sep 14 08:00:03.688956 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:03.688893    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-hllm7" podStartSLOduration=1.284529898 podStartE2EDuration="3.688875725s" totalImagesPullingTime="2.404345827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:03.686856272 +0000 UTC m=+724.315743924" watchObservedRunningTime="2026-09-14 08:00:03.688875725 +0000 UTC m=+724.317763352"
Sep 14 08:00:04.676378 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:04.676340    2563 generic.go:356] "Generic (PLEG): container finished" podID="68393035-a1a6-462e-b4a1-935a74ac0d71" containerID="5cf94aa4e00fe1abb0d76fd573ccaa435baba611678a1e1bfe5187449454c4a7" exitCode=0
Sep 14 08:00:04.676822 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:04.676427    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-hllm7" event={"ID":"68393035-a1a6-462e-b4a1-935a74ac0d71","Type":"ContainerDied","Data":"5cf94aa4e00fe1abb0d76fd573ccaa435baba611678a1e1bfe5187449454c4a7"}
Sep 14 08:00:05.802707 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:05.802681    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-hllm7"
Sep 14 08:00:06.688375 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:06.688345    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-hllm7"
Sep 14 08:00:06.688375 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:06.688358    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-hllm7" event={"ID":"68393035-a1a6-462e-b4a1-935a74ac0d71","Type":"ContainerDied","Data":"59f341e9b0f00e28ec31c83cf38f09542037b349ca4a2ef9d9b05e44e581d03c"}
Sep 14 08:00:06.688594 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:06.688399    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59f341e9b0f00e28ec31c83cf38f09542037b349ca4a2ef9d9b05e44e581d03c"
Sep 14 08:00:32.349595 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:32.349551    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7873-cccb64"]
Sep 14 08:00:32.350243 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:32.350171    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68393035-a1a6-462e-b4a1-935a74ac0d71" containerName="cleanup"
Sep 14 08:00:32.350243 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:32.350192    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="68393035-a1a6-462e-b4a1-935a74ac0d71" containerName="cleanup"
Sep 14 08:00:32.350364 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:32.350281    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="68393035-a1a6-462e-b4a1-935a74ac0d71" containerName="cleanup"
Sep 14 08:00:32.355654 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:32.355618    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7873-cccb64"
Sep 14 08:00:32.360409 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:32.360259    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fvblv\""
Sep 14 08:00:32.363483 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:32.363105    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7873-cccb64"]
Sep 14 08:00:32.404052 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:32.404023    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pldhv\" (UniqueName: \"kubernetes.io/projected/1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb-kube-api-access-pldhv\") pod \"test-curl-7873-cccb64\" (UID: \"1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb\") " pod="opendatahub/test-curl-7873-cccb64"
Sep 14 08:00:32.505087 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:32.505049    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pldhv\" (UniqueName: \"kubernetes.io/projected/1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb-kube-api-access-pldhv\") pod \"test-curl-7873-cccb64\" (UID: \"1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb\") " pod="opendatahub/test-curl-7873-cccb64"
Sep 14 08:00:32.514115 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:32.514084    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pldhv\" (UniqueName: \"kubernetes.io/projected/1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb-kube-api-access-pldhv\") pod \"test-curl-7873-cccb64\" (UID: \"1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb\") " pod="opendatahub/test-curl-7873-cccb64"
Sep 14 08:00:32.666416 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:32.666337    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7873-cccb64"
Sep 14 08:00:32.940144 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:32.940113    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7873-cccb64"]
Sep 14 08:00:32.941447 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:00:32.941417    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c3c20a8_cc09_4e75_ba9e_d8b5bd6791eb.slice/crio-2ca45907d386836625056739dbd4dfd1031db67129f25c0f68fdc9f486e44ee1 WatchSource:0}: Error finding container 2ca45907d386836625056739dbd4dfd1031db67129f25c0f68fdc9f486e44ee1: Status 404 returned error can't find the container with id 2ca45907d386836625056739dbd4dfd1031db67129f25c0f68fdc9f486e44ee1
Sep 14 08:00:33.847439 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:33.847394    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7873-cccb64" event={"ID":"1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb","Type":"ContainerStarted","Data":"2ca45907d386836625056739dbd4dfd1031db67129f25c0f68fdc9f486e44ee1"}
Sep 14 08:00:35.864455 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:35.864415    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7873-cccb64" event={"ID":"1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb","Type":"ContainerStarted","Data":"d129392092fb3d85446fe4887b2f660ba47ce98abc03b3288cd97e97346b7857"}
Sep 14 08:00:35.884214 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:35.884164    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7873-cccb64" podStartSLOduration=1.7307687999999999 podStartE2EDuration="3.88414845s" totalImagesPullingTime="2.15337965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:35.881606362 +0000 UTC m=+756.510493990" watchObservedRunningTime="2026-09-14 08:00:35.88414845 +0000 UTC m=+756.513036076"
Sep 14 08:00:36.485686 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.485655    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"]
Sep 14 08:00:36.489272 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.489253    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"
Sep 14 08:00:36.491576 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.491549    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-b69a34\""
Sep 14 08:00:36.497749 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.497726    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"]
Sep 14 08:00:36.503873 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.503847    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7873-cccb64"]
Sep 14 08:00:36.513748 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.513723    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7873-cccb64"]
Sep 14 08:00:36.544984 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.544899    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66336fc7-cbac-4183-afbf-d8e005cdcbbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k\" (UID: \"66336fc7-cbac-4183-afbf-d8e005cdcbbb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"
Sep 14 08:00:36.545146 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.545092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v7mp\" (UniqueName: \"kubernetes.io/projected/66336fc7-cbac-4183-afbf-d8e005cdcbbb-kube-api-access-4v7mp\") pod \"payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k\" (UID: \"66336fc7-cbac-4183-afbf-d8e005cdcbbb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"
Sep 14 08:00:36.623446 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.623414    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"]
Sep 14 08:00:36.623865 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.623848    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb" containerName="test-curl-7873-cccb64"
Sep 14 08:00:36.623924 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.623871    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb" containerName="test-curl-7873-cccb64"
Sep 14 08:00:36.623994 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.623982    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb" containerName="test-curl-7873-cccb64"
Sep 14 08:00:36.627048 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.627025    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"
Sep 14 08:00:36.629251 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.629230    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-b69a34-dockercfg-c265k\""
Sep 14 08:00:36.636846 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.636821    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"]
Sep 14 08:00:36.646200 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.646150    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcgss\" (UniqueName: \"kubernetes.io/projected/ab0c3b13-6781-4162-ac9c-85416ac28bae-kube-api-access-rcgss\") pod \"payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj\" (UID: \"ab0c3b13-6781-4162-ac9c-85416ac28bae\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"
Sep 14 08:00:36.646334 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.646232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v7mp\" (UniqueName: \"kubernetes.io/projected/66336fc7-cbac-4183-afbf-d8e005cdcbbb-kube-api-access-4v7mp\") pod \"payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k\" (UID: \"66336fc7-cbac-4183-afbf-d8e005cdcbbb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"
Sep 14 08:00:36.646405 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.646349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66336fc7-cbac-4183-afbf-d8e005cdcbbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k\" (UID: \"66336fc7-cbac-4183-afbf-d8e005cdcbbb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"
Sep 14 08:00:36.646405 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.646382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab0c3b13-6781-4162-ac9c-85416ac28bae-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj\" (UID: \"ab0c3b13-6781-4162-ac9c-85416ac28bae\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"
Sep 14 08:00:36.647422 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.647397    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66336fc7-cbac-4183-afbf-d8e005cdcbbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k\" (UID: \"66336fc7-cbac-4183-afbf-d8e005cdcbbb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"
Sep 14 08:00:36.654630 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.654604    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v7mp\" (UniqueName: \"kubernetes.io/projected/66336fc7-cbac-4183-afbf-d8e005cdcbbb-kube-api-access-4v7mp\") pod \"payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k\" (UID: \"66336fc7-cbac-4183-afbf-d8e005cdcbbb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"
Sep 14 08:00:36.747341 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.747279    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcgss\" (UniqueName: \"kubernetes.io/projected/ab0c3b13-6781-4162-ac9c-85416ac28bae-kube-api-access-rcgss\") pod \"payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj\" (UID: \"ab0c3b13-6781-4162-ac9c-85416ac28bae\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"
Sep 14 08:00:36.747510 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.747419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab0c3b13-6781-4162-ac9c-85416ac28bae-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj\" (UID: \"ab0c3b13-6781-4162-ac9c-85416ac28bae\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"
Sep 14 08:00:36.748200 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.748168    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab0c3b13-6781-4162-ac9c-85416ac28bae-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj\" (UID: \"ab0c3b13-6781-4162-ac9c-85416ac28bae\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"
Sep 14 08:00:36.756134 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.756110    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcgss\" (UniqueName: \"kubernetes.io/projected/ab0c3b13-6781-4162-ac9c-85416ac28bae-kube-api-access-rcgss\") pod \"payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj\" (UID: \"ab0c3b13-6781-4162-ac9c-85416ac28bae\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"
Sep 14 08:00:36.801295 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.801224    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"
Sep 14 08:00:36.877677 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:00:36.876048    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7873-cccb64" container="test-curl-7873-cccb64"
Sep 14 08:00:36.939678 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:36.939608    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"
Sep 14 08:00:37.088961 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:00:37.088925    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66336fc7_cbac_4183_afbf_d8e005cdcbbb.slice/crio-538dd4050f3c7dc60223023ab9700e89936f9063b677cd77af71132a991d9acc WatchSource:0}: Error finding container 538dd4050f3c7dc60223023ab9700e89936f9063b677cd77af71132a991d9acc: Status 404 returned error can't find the container with id 538dd4050f3c7dc60223023ab9700e89936f9063b677cd77af71132a991d9acc
Sep 14 08:00:37.094949 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.094927    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"]
Sep 14 08:00:37.224905 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.224876    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"]
Sep 14 08:00:37.225730 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:00:37.225693    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab0c3b13_6781_4162_ac9c_85416ac28bae.slice/crio-7ca2e77a501978f53cbaf75d9227096f8010c932089750e297b38053048b1671 WatchSource:0}: Error finding container 7ca2e77a501978f53cbaf75d9227096f8010c932089750e297b38053048b1671: Status 404 returned error can't find the container with id 7ca2e77a501978f53cbaf75d9227096f8010c932089750e297b38053048b1671
Sep 14 08:00:37.666867 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.666824    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"]
Sep 14 08:00:37.670599 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.670578    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"
Sep 14 08:00:37.673211 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.673182    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-814672-dockercfg-rgm4n\""
Sep 14 08:00:37.673211 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.673198    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-814672\""
Sep 14 08:00:37.682530 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.682506    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"]
Sep 14 08:00:37.710486 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.710430    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"]
Sep 14 08:00:37.714196 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.714171    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"
Sep 14 08:00:37.727636 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.727610    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"]
Sep 14 08:00:37.758864 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.758794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhj6w\" (UniqueName: \"kubernetes.io/projected/6c78d8a3-2056-488e-a72e-f05a727c54d7-kube-api-access-xhj6w\") pod \"payload-processing-e2e-worker-w0-814672-74d9548755-hpck9\" (UID: \"6c78d8a3-2056-488e-a72e-f05a727c54d7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"
Sep 14 08:00:37.758864 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.758848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xncrw\" (UniqueName: \"kubernetes.io/projected/78e9b78a-6eeb-4c01-a01c-beb29edf387b-kube-api-access-xncrw\") pod \"payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h\" (UID: \"78e9b78a-6eeb-4c01-a01c-beb29edf387b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"
Sep 14 08:00:37.759123 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.758880    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c78d8a3-2056-488e-a72e-f05a727c54d7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-814672-74d9548755-hpck9\" (UID: \"6c78d8a3-2056-488e-a72e-f05a727c54d7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"
Sep 14 08:00:37.759123 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.758918    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78e9b78a-6eeb-4c01-a01c-beb29edf387b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h\" (UID: \"78e9b78a-6eeb-4c01-a01c-beb29edf387b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"
Sep 14 08:00:37.860417 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.860377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78e9b78a-6eeb-4c01-a01c-beb29edf387b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h\" (UID: \"78e9b78a-6eeb-4c01-a01c-beb29edf387b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"
Sep 14 08:00:37.860606 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.860515    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhj6w\" (UniqueName: \"kubernetes.io/projected/6c78d8a3-2056-488e-a72e-f05a727c54d7-kube-api-access-xhj6w\") pod \"payload-processing-e2e-worker-w0-814672-74d9548755-hpck9\" (UID: \"6c78d8a3-2056-488e-a72e-f05a727c54d7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"
Sep 14 08:00:37.860606 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.860552    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xncrw\" (UniqueName: \"kubernetes.io/projected/78e9b78a-6eeb-4c01-a01c-beb29edf387b-kube-api-access-xncrw\") pod \"payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h\" (UID: \"78e9b78a-6eeb-4c01-a01c-beb29edf387b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"
Sep 14 08:00:37.860606 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.860577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c78d8a3-2056-488e-a72e-f05a727c54d7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-814672-74d9548755-hpck9\" (UID: \"6c78d8a3-2056-488e-a72e-f05a727c54d7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"
Sep 14 08:00:37.861314 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.861281    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78e9b78a-6eeb-4c01-a01c-beb29edf387b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h\" (UID: \"78e9b78a-6eeb-4c01-a01c-beb29edf387b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"
Sep 14 08:00:37.861454 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.861419    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c78d8a3-2056-488e-a72e-f05a727c54d7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-814672-74d9548755-hpck9\" (UID: \"6c78d8a3-2056-488e-a72e-f05a727c54d7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"
Sep 14 08:00:37.873312 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.873286    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhj6w\" (UniqueName: \"kubernetes.io/projected/6c78d8a3-2056-488e-a72e-f05a727c54d7-kube-api-access-xhj6w\") pod \"payload-processing-e2e-worker-w0-814672-74d9548755-hpck9\" (UID: \"6c78d8a3-2056-488e-a72e-f05a727c54d7\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"
Sep 14 08:00:37.877865 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.877839    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xncrw\" (UniqueName: \"kubernetes.io/projected/78e9b78a-6eeb-4c01-a01c-beb29edf387b-kube-api-access-xncrw\") pod \"payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h\" (UID: \"78e9b78a-6eeb-4c01-a01c-beb29edf387b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"
Sep 14 08:00:37.886944 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.886907    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7873-cccb64" podUID="1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb" containerName="test-curl-7873-cccb64" containerID="cri-o://d129392092fb3d85446fe4887b2f660ba47ce98abc03b3288cd97e97346b7857" gracePeriod=2
Sep 14 08:00:37.887758 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.887728    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"
Sep 14 08:00:37.887861 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.887770    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k" event={"ID":"66336fc7-cbac-4183-afbf-d8e005cdcbbb","Type":"ContainerStarted","Data":"1eeeb28d551dce83e75680137740fc255ac61b1128fdbc01ecc80f7574920d30"}
Sep 14 08:00:37.887861 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.887791    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"
Sep 14 08:00:37.887861 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.887806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k" event={"ID":"66336fc7-cbac-4183-afbf-d8e005cdcbbb","Type":"ContainerStarted","Data":"538dd4050f3c7dc60223023ab9700e89936f9063b677cd77af71132a991d9acc"}
Sep 14 08:00:37.887861 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.887819    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj" event={"ID":"ab0c3b13-6781-4162-ac9c-85416ac28bae","Type":"ContainerStarted","Data":"57e06e369968ceb7a5ccd7750c4d874d0a7e6c95d6fa4b339a30de9dab3f6fc1"}
Sep 14 08:00:37.887861 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.887835    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj" event={"ID":"ab0c3b13-6781-4162-ac9c-85416ac28bae","Type":"ContainerStarted","Data":"7ca2e77a501978f53cbaf75d9227096f8010c932089750e297b38053048b1671"}
Sep 14 08:00:37.920992 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.920905    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k" podStartSLOduration=1.920884267 podStartE2EDuration="1.920884267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:37.917730521 +0000 UTC m=+758.546618184" watchObservedRunningTime="2026-09-14 08:00:37.920884267 +0000 UTC m=+758.549771897"
Sep 14 08:00:37.955146 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.955091    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj" podStartSLOduration=1.955075935 podStartE2EDuration="1.955075935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:37.955011861 +0000 UTC m=+758.583899507" watchObservedRunningTime="2026-09-14 08:00:37.955075935 +0000 UTC m=+758.583963563"
Sep 14 08:00:37.981817 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:37.981786    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"
Sep 14 08:00:38.025001 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.024960    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"
Sep 14 08:00:38.272451 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.272138    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"]
Sep 14 08:00:38.275318 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:00:38.275293    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c78d8a3_2056_488e_a72e_f05a727c54d7.slice/crio-58a85b00d8798069f5859b180d013151fc93f7a619af7492b4d16bd207cdd6be WatchSource:0}: Error finding container 58a85b00d8798069f5859b180d013151fc93f7a619af7492b4d16bd207cdd6be: Status 404 returned error can't find the container with id 58a85b00d8798069f5859b180d013151fc93f7a619af7492b4d16bd207cdd6be
Sep 14 08:00:38.322709 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.322685    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"]
Sep 14 08:00:38.325809 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:00:38.325784    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78e9b78a_6eeb_4c01_a01c_beb29edf387b.slice/crio-99e710fd54ef686d1674a21f09533e02a7f3d8f2cd987bbc8499e757653148a2 WatchSource:0}: Error finding container 99e710fd54ef686d1674a21f09533e02a7f3d8f2cd987bbc8499e757653148a2: Status 404 returned error can't find the container with id 99e710fd54ef686d1674a21f09533e02a7f3d8f2cd987bbc8499e757653148a2
Sep 14 08:00:38.839403 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.839372    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"]
Sep 14 08:00:38.843764 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.843739    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"
Sep 14 08:00:38.847468 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.847281    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-3a842e\""
Sep 14 08:00:38.858101 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.858075    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"]
Sep 14 08:00:38.861853 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.861825    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"]
Sep 14 08:00:38.868203 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.868176    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"
Sep 14 08:00:38.871120 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.871096    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-3a842e-dockercfg-fbfcg\""
Sep 14 08:00:38.871407 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.871376    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63b131c7-8203-411e-ae1e-bf139fe6a246-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm\" (UID: \"63b131c7-8203-411e-ae1e-bf139fe6a246\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"
Sep 14 08:00:38.871709 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.871686    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zh5c\" (UniqueName: \"kubernetes.io/projected/63b131c7-8203-411e-ae1e-bf139fe6a246-kube-api-access-4zh5c\") pod \"payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm\" (UID: \"63b131c7-8203-411e-ae1e-bf139fe6a246\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"
Sep 14 08:00:38.871800 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.871689    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"]
Sep 14 08:00:38.894778 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.894744    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9" event={"ID":"6c78d8a3-2056-488e-a72e-f05a727c54d7","Type":"ContainerStarted","Data":"1c1d7f27465aa3e190d415ed8c4988f21b15000a7416b9d07db411cff97fdb8d"}
Sep 14 08:00:38.895210 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.894806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9" event={"ID":"6c78d8a3-2056-488e-a72e-f05a727c54d7","Type":"ContainerStarted","Data":"58a85b00d8798069f5859b180d013151fc93f7a619af7492b4d16bd207cdd6be"}
Sep 14 08:00:38.895210 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.894973    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"
Sep 14 08:00:38.897561 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.897534    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h" event={"ID":"78e9b78a-6eeb-4c01-a01c-beb29edf387b","Type":"ContainerStarted","Data":"9a70f615b878e4be5935f9fa88713b007e7b9ee1f8948001dce968a2450b9c85"}
Sep 14 08:00:38.897696 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.897582    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h" event={"ID":"78e9b78a-6eeb-4c01-a01c-beb29edf387b","Type":"ContainerStarted","Data":"99e710fd54ef686d1674a21f09533e02a7f3d8f2cd987bbc8499e757653148a2"}
Sep 14 08:00:38.898000 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.897982    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"
Sep 14 08:00:38.912662 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.912593    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9" podStartSLOduration=1.912578672 podStartE2EDuration="1.912578672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:38.911758379 +0000 UTC m=+759.540646005" watchObservedRunningTime="2026-09-14 08:00:38.912578672 +0000 UTC m=+759.541466299"
Sep 14 08:00:38.929438 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.929397    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h" podStartSLOduration=1.9293838810000001 podStartE2EDuration="1.929383881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:38.927983462 +0000 UTC m=+759.556871113" watchObservedRunningTime="2026-09-14 08:00:38.929383881 +0000 UTC m=+759.558271502"
Sep 14 08:00:38.972297 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.972256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mkg2\" (UniqueName: \"kubernetes.io/projected/c4dc89e9-1855-4713-8915-b553d65aee4e-kube-api-access-8mkg2\") pod \"payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2\" (UID: \"c4dc89e9-1855-4713-8915-b553d65aee4e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"
Sep 14 08:00:38.972475 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.972303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zh5c\" (UniqueName: \"kubernetes.io/projected/63b131c7-8203-411e-ae1e-bf139fe6a246-kube-api-access-4zh5c\") pod \"payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm\" (UID: \"63b131c7-8203-411e-ae1e-bf139fe6a246\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"
Sep 14 08:00:38.972475 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.972371    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63b131c7-8203-411e-ae1e-bf139fe6a246-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm\" (UID: \"63b131c7-8203-411e-ae1e-bf139fe6a246\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"
Sep 14 08:00:38.972475 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.972458    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4dc89e9-1855-4713-8915-b553d65aee4e-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2\" (UID: \"c4dc89e9-1855-4713-8915-b553d65aee4e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"
Sep 14 08:00:38.973776 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.973747    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63b131c7-8203-411e-ae1e-bf139fe6a246-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm\" (UID: \"63b131c7-8203-411e-ae1e-bf139fe6a246\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"
Sep 14 08:00:38.981719 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:38.981695    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zh5c\" (UniqueName: \"kubernetes.io/projected/63b131c7-8203-411e-ae1e-bf139fe6a246-kube-api-access-4zh5c\") pod \"payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm\" (UID: \"63b131c7-8203-411e-ae1e-bf139fe6a246\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"
Sep 14 08:00:39.073100 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.073059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4dc89e9-1855-4713-8915-b553d65aee4e-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2\" (UID: \"c4dc89e9-1855-4713-8915-b553d65aee4e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"
Sep 14 08:00:39.073302 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.073181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mkg2\" (UniqueName: \"kubernetes.io/projected/c4dc89e9-1855-4713-8915-b553d65aee4e-kube-api-access-8mkg2\") pod \"payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2\" (UID: \"c4dc89e9-1855-4713-8915-b553d65aee4e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"
Sep 14 08:00:39.074016 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.073986    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4dc89e9-1855-4713-8915-b553d65aee4e-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2\" (UID: \"c4dc89e9-1855-4713-8915-b553d65aee4e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"
Sep 14 08:00:39.082765 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.082739    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mkg2\" (UniqueName: \"kubernetes.io/projected/c4dc89e9-1855-4713-8915-b553d65aee4e-kube-api-access-8mkg2\") pod \"payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2\" (UID: \"c4dc89e9-1855-4713-8915-b553d65aee4e\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"
Sep 14 08:00:39.168609 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.168519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"
Sep 14 08:00:39.180801 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.180774    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"
Sep 14 08:00:39.449232 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.449204    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"]
Sep 14 08:00:39.451007 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:00:39.450965    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b131c7_8203_411e_ae1e_bf139fe6a246.slice/crio-33e92aaa1601984eafb93029fad77611811237adbdf33f263c09e4c43d1915a0 WatchSource:0}: Error finding container 33e92aaa1601984eafb93029fad77611811237adbdf33f263c09e4c43d1915a0: Status 404 returned error can't find the container with id 33e92aaa1601984eafb93029fad77611811237adbdf33f263c09e4c43d1915a0
Sep 14 08:00:39.483534 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.483507    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"]
Sep 14 08:00:39.485934 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:00:39.485908    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4dc89e9_1855_4713_8915_b553d65aee4e.slice/crio-f27d42011269741619b039072a16d62882cf75c72096741fd4f20db763871ae8 WatchSource:0}: Error finding container f27d42011269741619b039072a16d62882cf75c72096741fd4f20db763871ae8: Status 404 returned error can't find the container with id f27d42011269741619b039072a16d62882cf75c72096741fd4f20db763871ae8
Sep 14 08:00:39.911512 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.911480    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm" event={"ID":"63b131c7-8203-411e-ae1e-bf139fe6a246","Type":"ContainerStarted","Data":"b8f0c7b55c2ebed36eaa5cec1685182d58cce2d4c2a2ec86689a62709dd4aa74"}
Sep 14 08:00:39.911825 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.911543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm" event={"ID":"63b131c7-8203-411e-ae1e-bf139fe6a246","Type":"ContainerStarted","Data":"33e92aaa1601984eafb93029fad77611811237adbdf33f263c09e4c43d1915a0"}
Sep 14 08:00:39.911825 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.911574    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"
Sep 14 08:00:39.913722 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.913700    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2" event={"ID":"c4dc89e9-1855-4713-8915-b553d65aee4e","Type":"ContainerStarted","Data":"b356ea9de3205c927d001e5bdde5b501bf4aca77ab5dd9eb081fe8aef662ca71"}
Sep 14 08:00:39.913874 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.913842    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2" event={"ID":"c4dc89e9-1855-4713-8915-b553d65aee4e","Type":"ContainerStarted","Data":"f27d42011269741619b039072a16d62882cf75c72096741fd4f20db763871ae8"}
Sep 14 08:00:39.914202 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.914181    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"
Sep 14 08:00:39.953605 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.953553    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm" podStartSLOduration=1.95353062 podStartE2EDuration="1.95353062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:39.950830791 +0000 UTC m=+760.579718447" watchObservedRunningTime="2026-09-14 08:00:39.95353062 +0000 UTC m=+760.582418246"
Sep 14 08:00:39.988218 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:39.988168    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2" podStartSLOduration=1.988145941 podStartE2EDuration="1.988145941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:39.98777143 +0000 UTC m=+760.616659083" watchObservedRunningTime="2026-09-14 08:00:39.988145941 +0000 UTC m=+760.617033568"
Sep 14 08:00:40.130201 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:40.130176    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7873-cccb64"
Sep 14 08:00:40.132053 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:00:40.132026    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7873-cccb64\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb" pod="opendatahub/test-curl-7873-cccb64"
Sep 14 08:00:40.183587 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:40.183502    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pldhv\" (UniqueName: \"kubernetes.io/projected/1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb-kube-api-access-pldhv\") pod \"1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb\" (UID: \"1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb\") "
Sep 14 08:00:40.185843 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:40.185806    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb-kube-api-access-pldhv" (OuterVolumeSpecName: "kube-api-access-pldhv") pod "1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb" (UID: "1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb"). InnerVolumeSpecName "kube-api-access-pldhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:00:40.285321 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:40.285289    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pldhv\" (UniqueName: \"kubernetes.io/projected/1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb-kube-api-access-pldhv\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:00:40.921473 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:40.921435    2563 generic.go:356] "Generic (PLEG): container finished" podID="1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb" containerID="d129392092fb3d85446fe4887b2f660ba47ce98abc03b3288cd97e97346b7857" exitCode=137
Sep 14 08:00:40.921982 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:40.921477    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7873-cccb64"
Sep 14 08:00:40.921982 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:40.921538    2563 scope.go:117] "RemoveContainer" containerID="d129392092fb3d85446fe4887b2f660ba47ce98abc03b3288cd97e97346b7857"
Sep 14 08:00:40.924060 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:00:40.924038    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7873-cccb64\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb" pod="opendatahub/test-curl-7873-cccb64"
Sep 14 08:00:40.932398 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:40.932378    2563 scope.go:117] "RemoveContainer" containerID="d129392092fb3d85446fe4887b2f660ba47ce98abc03b3288cd97e97346b7857"
Sep 14 08:00:40.932685 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:00:40.932659    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d129392092fb3d85446fe4887b2f660ba47ce98abc03b3288cd97e97346b7857\": container with ID starting with d129392092fb3d85446fe4887b2f660ba47ce98abc03b3288cd97e97346b7857 not found: ID does not exist" containerID="d129392092fb3d85446fe4887b2f660ba47ce98abc03b3288cd97e97346b7857"
Sep 14 08:00:40.932768 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:40.932693    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d129392092fb3d85446fe4887b2f660ba47ce98abc03b3288cd97e97346b7857"} err="failed to get container status \"d129392092fb3d85446fe4887b2f660ba47ce98abc03b3288cd97e97346b7857\": rpc error: code = NotFound desc = could not find container \"d129392092fb3d85446fe4887b2f660ba47ce98abc03b3288cd97e97346b7857\": container with ID starting with d129392092fb3d85446fe4887b2f660ba47ce98abc03b3288cd97e97346b7857 not found: ID does not exist"
Sep 14 08:00:40.949396 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:00:40.949370    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7873-cccb64\" is forbidden: User \"system:node:ip-10-0-143-250.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-250.ec2.internal' and this object" podUID="1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb" pod="opendatahub/test-curl-7873-cccb64"
Sep 14 08:00:41.885759 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:41.885725    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb" path="/var/lib/kubelet/pods/1c3c20a8-cc09-4e75-ba9e-d8b5bd6791eb/volumes"
Sep 14 08:00:45.139980 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.139942    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"]
Sep 14 08:00:45.144987 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.144962    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"
Sep 14 08:00:45.147469 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.147448    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-22b517\""
Sep 14 08:00:45.155623 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.155590    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"]
Sep 14 08:00:45.205826 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.205794    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"]
Sep 14 08:00:45.209447 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.209429    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"
Sep 14 08:00:45.212012 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.211993    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-22b517-dockercfg-z54rn\""
Sep 14 08:00:45.220738 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.220713    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"]
Sep 14 08:00:45.232687 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.232631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e49d370-ad30-4bc5-a780-e77432baa704-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp\" (UID: \"8e49d370-ad30-4bc5-a780-e77432baa704\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"
Sep 14 08:00:45.232830 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.232793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfgkq\" (UniqueName: \"kubernetes.io/projected/8e49d370-ad30-4bc5-a780-e77432baa704-kube-api-access-qfgkq\") pod \"payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp\" (UID: \"8e49d370-ad30-4bc5-a780-e77432baa704\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"
Sep 14 08:00:45.333535 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.333503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfwg6\" (UniqueName: \"kubernetes.io/projected/9d8a82c9-6642-4ee6-bcb4-dc08071b8207-kube-api-access-zfwg6\") pod \"payload-processing-e2e-autores-22b517-79f754b75-brgfw\" (UID: \"9d8a82c9-6642-4ee6-bcb4-dc08071b8207\") " pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"
Sep 14 08:00:45.333711 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.333560    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfgkq\" (UniqueName: \"kubernetes.io/projected/8e49d370-ad30-4bc5-a780-e77432baa704-kube-api-access-qfgkq\") pod \"payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp\" (UID: \"8e49d370-ad30-4bc5-a780-e77432baa704\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"
Sep 14 08:00:45.333711 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.333695    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e49d370-ad30-4bc5-a780-e77432baa704-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp\" (UID: \"8e49d370-ad30-4bc5-a780-e77432baa704\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"
Sep 14 08:00:45.333792 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.333751    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8a82c9-6642-4ee6-bcb4-dc08071b8207-plugins-config-volume\") pod \"payload-processing-e2e-autores-22b517-79f754b75-brgfw\" (UID: \"9d8a82c9-6642-4ee6-bcb4-dc08071b8207\") " pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"
Sep 14 08:00:45.334430 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.334407    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e49d370-ad30-4bc5-a780-e77432baa704-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp\" (UID: \"8e49d370-ad30-4bc5-a780-e77432baa704\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"
Sep 14 08:00:45.343559 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.343533    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfgkq\" (UniqueName: \"kubernetes.io/projected/8e49d370-ad30-4bc5-a780-e77432baa704-kube-api-access-qfgkq\") pod \"payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp\" (UID: \"8e49d370-ad30-4bc5-a780-e77432baa704\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"
Sep 14 08:00:45.435040 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.434955    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8a82c9-6642-4ee6-bcb4-dc08071b8207-plugins-config-volume\") pod \"payload-processing-e2e-autores-22b517-79f754b75-brgfw\" (UID: \"9d8a82c9-6642-4ee6-bcb4-dc08071b8207\") " pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"
Sep 14 08:00:45.435040 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.435008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfwg6\" (UniqueName: \"kubernetes.io/projected/9d8a82c9-6642-4ee6-bcb4-dc08071b8207-kube-api-access-zfwg6\") pod \"payload-processing-e2e-autores-22b517-79f754b75-brgfw\" (UID: \"9d8a82c9-6642-4ee6-bcb4-dc08071b8207\") " pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"
Sep 14 08:00:45.435743 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.435724    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8a82c9-6642-4ee6-bcb4-dc08071b8207-plugins-config-volume\") pod \"payload-processing-e2e-autores-22b517-79f754b75-brgfw\" (UID: \"9d8a82c9-6642-4ee6-bcb4-dc08071b8207\") " pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"
Sep 14 08:00:45.445411 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.445380    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfwg6\" (UniqueName: \"kubernetes.io/projected/9d8a82c9-6642-4ee6-bcb4-dc08071b8207-kube-api-access-zfwg6\") pod \"payload-processing-e2e-autores-22b517-79f754b75-brgfw\" (UID: \"9d8a82c9-6642-4ee6-bcb4-dc08071b8207\") " pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"
Sep 14 08:00:45.457033 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.457004    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"
Sep 14 08:00:45.520414 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.520383    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"
Sep 14 08:00:45.745479 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.745446    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"]
Sep 14 08:00:45.746789 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:00:45.746754    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e49d370_ad30_4bc5_a780_e77432baa704.slice/crio-012e57a00a4aa509138be2893e017e4d3f29726a7166472ef55e7d2f503ec704 WatchSource:0}: Error finding container 012e57a00a4aa509138be2893e017e4d3f29726a7166472ef55e7d2f503ec704: Status 404 returned error can't find the container with id 012e57a00a4aa509138be2893e017e4d3f29726a7166472ef55e7d2f503ec704
Sep 14 08:00:45.805592 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.805567    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"]
Sep 14 08:00:45.806386 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:00:45.806366    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d8a82c9_6642_4ee6_bcb4_dc08071b8207.slice/crio-eb85e59293b756678b90e11ae0b23cd1f87e11b5486ca13e4107d9b3c6be1b1c WatchSource:0}: Error finding container eb85e59293b756678b90e11ae0b23cd1f87e11b5486ca13e4107d9b3c6be1b1c: Status 404 returned error can't find the container with id eb85e59293b756678b90e11ae0b23cd1f87e11b5486ca13e4107d9b3c6be1b1c
Sep 14 08:00:45.962517 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.962482    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw" event={"ID":"9d8a82c9-6642-4ee6-bcb4-dc08071b8207","Type":"ContainerStarted","Data":"2f0309a9fed2e63d9120eebad6d929d01abe458a4121cd74aee83f5ae78fb393"}
Sep 14 08:00:45.962742 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.962546    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw" event={"ID":"9d8a82c9-6642-4ee6-bcb4-dc08071b8207","Type":"ContainerStarted","Data":"eb85e59293b756678b90e11ae0b23cd1f87e11b5486ca13e4107d9b3c6be1b1c"}
Sep 14 08:00:45.962742 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.962580    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"
Sep 14 08:00:45.964434 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.964403    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp" event={"ID":"8e49d370-ad30-4bc5-a780-e77432baa704","Type":"ContainerStarted","Data":"0e2d5041ea5b110f30080e99b216eb480e44b04020d71994cf803053aeb4adbc"}
Sep 14 08:00:45.964569 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.964455    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp" event={"ID":"8e49d370-ad30-4bc5-a780-e77432baa704","Type":"ContainerStarted","Data":"012e57a00a4aa509138be2893e017e4d3f29726a7166472ef55e7d2f503ec704"}
Sep 14 08:00:45.964569 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.964530    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"
Sep 14 08:00:45.984267 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:45.984210    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw" podStartSLOduration=0.984188759 podStartE2EDuration="984.188759ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:45.980602019 +0000 UTC m=+766.609489667" watchObservedRunningTime="2026-09-14 08:00:45.984188759 +0000 UTC m=+766.613076389"
Sep 14 08:00:46.000295 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:46.000198    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp" podStartSLOduration=1.000177857 podStartE2EDuration="1.000177857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:45.997627524 +0000 UTC m=+766.626515169" watchObservedRunningTime="2026-09-14 08:00:46.000177857 +0000 UTC m=+766.629065522"
Sep 14 08:00:48.900217 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:48.900186    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"
Sep 14 08:00:49.147054 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.147016    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"]
Sep 14 08:00:49.150924 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.150869    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"
Sep 14 08:00:49.153997 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.153978    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a9d21f4e-dockercfg-bgkh9\""
Sep 14 08:00:49.154344 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.154315    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a9d21f4e\""
Sep 14 08:00:49.163577 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.163551    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"]
Sep 14 08:00:49.279813 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.279779    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k9d4\" (UniqueName: \"kubernetes.io/projected/5438f7ab-ec36-4eaf-a864-0253f9c631b3-kube-api-access-9k9d4\") pod \"payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v\" (UID: \"5438f7ab-ec36-4eaf-a864-0253f9c631b3\") " pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"
Sep 14 08:00:49.279961 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.279843    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5438f7ab-ec36-4eaf-a864-0253f9c631b3-plugins-config-volume\") pod \"payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v\" (UID: \"5438f7ab-ec36-4eaf-a864-0253f9c631b3\") " pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"
Sep 14 08:00:49.380783 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.380745    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k9d4\" (UniqueName: \"kubernetes.io/projected/5438f7ab-ec36-4eaf-a864-0253f9c631b3-kube-api-access-9k9d4\") pod \"payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v\" (UID: \"5438f7ab-ec36-4eaf-a864-0253f9c631b3\") " pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"
Sep 14 08:00:49.380956 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.380813    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5438f7ab-ec36-4eaf-a864-0253f9c631b3-plugins-config-volume\") pod \"payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v\" (UID: \"5438f7ab-ec36-4eaf-a864-0253f9c631b3\") " pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"
Sep 14 08:00:49.381597 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.381569    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5438f7ab-ec36-4eaf-a864-0253f9c631b3-plugins-config-volume\") pod \"payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v\" (UID: \"5438f7ab-ec36-4eaf-a864-0253f9c631b3\") " pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"
Sep 14 08:00:49.389999 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.389973    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k9d4\" (UniqueName: \"kubernetes.io/projected/5438f7ab-ec36-4eaf-a864-0253f9c631b3-kube-api-access-9k9d4\") pod \"payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v\" (UID: \"5438f7ab-ec36-4eaf-a864-0253f9c631b3\") " pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"
Sep 14 08:00:49.463918 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.463837    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"
Sep 14 08:00:49.744812 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.744785    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"]
Sep 14 08:00:49.746731 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:00:49.746704    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5438f7ab_ec36_4eaf_a864_0253f9c631b3.slice/crio-05933906d6577e72b0c65f293fca428440b6418d9674ced9619ed55fe7962240 WatchSource:0}: Error finding container 05933906d6577e72b0c65f293fca428440b6418d9674ced9619ed55fe7962240: Status 404 returned error can't find the container with id 05933906d6577e72b0c65f293fca428440b6418d9674ced9619ed55fe7962240
Sep 14 08:00:49.916051 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.916020    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"
Sep 14 08:00:49.916478 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.916099    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"
Sep 14 08:00:49.916478 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.916200    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"
Sep 14 08:00:49.993037 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.992934    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v" event={"ID":"5438f7ab-ec36-4eaf-a864-0253f9c631b3","Type":"ContainerStarted","Data":"480d666f760faa46f0db30866577512cd0ca0d78db9a8a97ac04fe034b6ee64c"}
Sep 14 08:00:49.993203 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.993040    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"
Sep 14 08:00:49.993203 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:49.993064    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v" event={"ID":"5438f7ab-ec36-4eaf-a864-0253f9c631b3","Type":"ContainerStarted","Data":"05933906d6577e72b0c65f293fca428440b6418d9674ced9619ed55fe7962240"}
Sep 14 08:00:50.013352 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:50.013297    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v" podStartSLOduration=1.013275288 podStartE2EDuration="1.013275288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:50.012837094 +0000 UTC m=+770.641724745" watchObservedRunningTime="2026-09-14 08:00:50.013275288 +0000 UTC m=+770.642162910"
Sep 14 08:00:50.924080 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:50.924046    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"
Sep 14 08:00:50.924430 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:50.924217    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"
Sep 14 08:00:56.972406 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:56.972369    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"
Sep 14 08:00:56.973023 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:00:56.972994    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"
Sep 14 08:01:01.011225 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:01.011192    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"
Sep 14 08:01:04.290079 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:04.290041    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm"]
Sep 14 08:01:04.297852 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:04.297833    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm"
Sep 14 08:01:04.300725 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:04.300698    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:01:04.303855 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:04.303829    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm"]
Sep 14 08:01:04.423937 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:04.423905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aba049e-68e3-4a5e-a7af-e604cc3e8124-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm\" (UID: \"1aba049e-68e3-4a5e-a7af-e604cc3e8124\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm"
Sep 14 08:01:04.524891 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:04.524856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aba049e-68e3-4a5e-a7af-e604cc3e8124-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm\" (UID: \"1aba049e-68e3-4a5e-a7af-e604cc3e8124\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm"
Sep 14 08:01:04.525465 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:04.525446    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aba049e-68e3-4a5e-a7af-e604cc3e8124-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm\" (UID: \"1aba049e-68e3-4a5e-a7af-e604cc3e8124\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm"
Sep 14 08:01:04.609861 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:04.609823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm"
Sep 14 08:01:04.901806 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:04.901779    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm"]
Sep 14 08:01:04.902779 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:01:04.902752    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aba049e_68e3_4a5e_a7af_e604cc3e8124.slice/crio-93c1d9d92d639188d468054af6efd9d3434bf115da592326fb2f9830a005e8dd WatchSource:0}: Error finding container 93c1d9d92d639188d468054af6efd9d3434bf115da592326fb2f9830a005e8dd: Status 404 returned error can't find the container with id 93c1d9d92d639188d468054af6efd9d3434bf115da592326fb2f9830a005e8dd
Sep 14 08:01:05.098184 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:05.098147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm" event={"ID":"1aba049e-68e3-4a5e-a7af-e604cc3e8124","Type":"ContainerStarted","Data":"93c1d9d92d639188d468054af6efd9d3434bf115da592326fb2f9830a005e8dd"}
Sep 14 08:01:06.106222 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.106185    2563 generic.go:356] "Generic (PLEG): container finished" podID="1aba049e-68e3-4a5e-a7af-e604cc3e8124" containerID="782916ecb8af7ee1ac23391b50cf45fa26f53f61e301333125e817244bed4adc" exitCode=0
Sep 14 08:01:06.106631 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.106223    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm" event={"ID":"1aba049e-68e3-4a5e-a7af-e604cc3e8124","Type":"ContainerDied","Data":"782916ecb8af7ee1ac23391b50cf45fa26f53f61e301333125e817244bed4adc"}
Sep 14 08:01:06.401958 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.401878    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"]
Sep 14 08:01:06.405724 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.405708    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"
Sep 14 08:01:06.408218 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.408198    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-5ae4e6\""
Sep 14 08:01:06.416594 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.416572    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"]
Sep 14 08:01:06.464636 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.464602    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"]
Sep 14 08:01:06.468285 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.468266    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"
Sep 14 08:01:06.471036 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.471017    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-5ae4e6-dockercfg-zgdfb\""
Sep 14 08:01:06.483737 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.483716    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"]
Sep 14 08:01:06.548272 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.548233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br4fb\" (UniqueName: \"kubernetes.io/projected/ae6dcda5-b5d5-454c-aefd-69b2457491b3-kube-api-access-br4fb\") pod \"payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd\" (UID: \"ae6dcda5-b5d5-454c-aefd-69b2457491b3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"
Sep 14 08:01:06.548272 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.548277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hc48\" (UniqueName: \"kubernetes.io/projected/95f9ae36-17dc-4c6d-9909-2bf2970d977f-kube-api-access-5hc48\") pod \"payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7\" (UID: \"95f9ae36-17dc-4c6d-9909-2bf2970d977f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"
Sep 14 08:01:06.548530 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.548300    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae6dcda5-b5d5-454c-aefd-69b2457491b3-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd\" (UID: \"ae6dcda5-b5d5-454c-aefd-69b2457491b3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"
Sep 14 08:01:06.548530 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.548413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95f9ae36-17dc-4c6d-9909-2bf2970d977f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7\" (UID: \"95f9ae36-17dc-4c6d-9909-2bf2970d977f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"
Sep 14 08:01:06.649782 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.649744    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br4fb\" (UniqueName: \"kubernetes.io/projected/ae6dcda5-b5d5-454c-aefd-69b2457491b3-kube-api-access-br4fb\") pod \"payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd\" (UID: \"ae6dcda5-b5d5-454c-aefd-69b2457491b3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"
Sep 14 08:01:06.649782 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.649787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hc48\" (UniqueName: \"kubernetes.io/projected/95f9ae36-17dc-4c6d-9909-2bf2970d977f-kube-api-access-5hc48\") pod \"payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7\" (UID: \"95f9ae36-17dc-4c6d-9909-2bf2970d977f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"
Sep 14 08:01:06.650042 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.649805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae6dcda5-b5d5-454c-aefd-69b2457491b3-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd\" (UID: \"ae6dcda5-b5d5-454c-aefd-69b2457491b3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"
Sep 14 08:01:06.650042 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.649837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95f9ae36-17dc-4c6d-9909-2bf2970d977f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7\" (UID: \"95f9ae36-17dc-4c6d-9909-2bf2970d977f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"
Sep 14 08:01:06.650530 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.650503    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95f9ae36-17dc-4c6d-9909-2bf2970d977f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7\" (UID: \"95f9ae36-17dc-4c6d-9909-2bf2970d977f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"
Sep 14 08:01:06.650628 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.650543    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae6dcda5-b5d5-454c-aefd-69b2457491b3-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd\" (UID: \"ae6dcda5-b5d5-454c-aefd-69b2457491b3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"
Sep 14 08:01:06.658842 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.658793    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br4fb\" (UniqueName: \"kubernetes.io/projected/ae6dcda5-b5d5-454c-aefd-69b2457491b3-kube-api-access-br4fb\") pod \"payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd\" (UID: \"ae6dcda5-b5d5-454c-aefd-69b2457491b3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"
Sep 14 08:01:06.658937 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.658921    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hc48\" (UniqueName: \"kubernetes.io/projected/95f9ae36-17dc-4c6d-9909-2bf2970d977f-kube-api-access-5hc48\") pod \"payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7\" (UID: \"95f9ae36-17dc-4c6d-9909-2bf2970d977f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"
Sep 14 08:01:06.716981 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.716956    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"
Sep 14 08:01:06.780463 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:06.779942    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"
Sep 14 08:01:07.000285 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:07.000252    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"]
Sep 14 08:01:07.002793 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:01:07.002750    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae6dcda5_b5d5_454c_aefd_69b2457491b3.slice/crio-943820aad6fa9b857a6ef84bf7e8b9b41deeab431ad366bc0059a524a426a855 WatchSource:0}: Error finding container 943820aad6fa9b857a6ef84bf7e8b9b41deeab431ad366bc0059a524a426a855: Status 404 returned error can't find the container with id 943820aad6fa9b857a6ef84bf7e8b9b41deeab431ad366bc0059a524a426a855
Sep 14 08:01:07.072360 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:07.072332    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"]
Sep 14 08:01:07.072972 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:01:07.072875    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95f9ae36_17dc_4c6d_9909_2bf2970d977f.slice/crio-6202a72e04c3193a4304e015c05128007ef84cebb3900a387fa45a6c8419522c WatchSource:0}: Error finding container 6202a72e04c3193a4304e015c05128007ef84cebb3900a387fa45a6c8419522c: Status 404 returned error can't find the container with id 6202a72e04c3193a4304e015c05128007ef84cebb3900a387fa45a6c8419522c
Sep 14 08:01:07.114974 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:07.114934    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd" event={"ID":"ae6dcda5-b5d5-454c-aefd-69b2457491b3","Type":"ContainerStarted","Data":"15bbd0bc413ae3b58052af72728496feaeae0940c7c4edd934224d15a82a31dc"}
Sep 14 08:01:07.115382 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:07.115001    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"
Sep 14 08:01:07.115382 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:07.115024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd" event={"ID":"ae6dcda5-b5d5-454c-aefd-69b2457491b3","Type":"ContainerStarted","Data":"943820aad6fa9b857a6ef84bf7e8b9b41deeab431ad366bc0059a524a426a855"}
Sep 14 08:01:07.116421 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:07.116398    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7" event={"ID":"95f9ae36-17dc-4c6d-9909-2bf2970d977f","Type":"ContainerStarted","Data":"6202a72e04c3193a4304e015c05128007ef84cebb3900a387fa45a6c8419522c"}
Sep 14 08:01:07.140196 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:07.139947    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd" podStartSLOduration=1.139922148 podStartE2EDuration="1.139922148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:01:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:01:07.136832214 +0000 UTC m=+787.765719841" watchObservedRunningTime="2026-09-14 08:01:07.139922148 +0000 UTC m=+787.768809777"
Sep 14 08:01:07.265424 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:07.257470    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm"
Sep 14 08:01:07.361079 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:07.361045    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aba049e-68e3-4a5e-a7af-e604cc3e8124-maas-api-service-ca\") pod \"1aba049e-68e3-4a5e-a7af-e604cc3e8124\" (UID: \"1aba049e-68e3-4a5e-a7af-e604cc3e8124\") "
Sep 14 08:01:07.361381 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:07.361358    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aba049e-68e3-4a5e-a7af-e604cc3e8124-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1aba049e-68e3-4a5e-a7af-e604cc3e8124" (UID: "1aba049e-68e3-4a5e-a7af-e604cc3e8124"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:01:07.462634 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:07.462589    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aba049e-68e3-4a5e-a7af-e604cc3e8124-maas-api-service-ca\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:01:08.123609 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:08.123574    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm"
Sep 14 08:01:08.123609 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:08.123596    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm" event={"ID":"1aba049e-68e3-4a5e-a7af-e604cc3e8124","Type":"ContainerDied","Data":"93c1d9d92d639188d468054af6efd9d3434bf115da592326fb2f9830a005e8dd"}
Sep 14 08:01:08.124090 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:08.123666    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="93c1d9d92d639188d468054af6efd9d3434bf115da592326fb2f9830a005e8dd"
Sep 14 08:01:08.125425 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:08.125395    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7" event={"ID":"95f9ae36-17dc-4c6d-9909-2bf2970d977f","Type":"ContainerStarted","Data":"ee2153af38e16907134ac3e154dc4416dcea3668167376ac321821f066eb1d4b"}
Sep 14 08:01:08.125553 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:08.125452    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"
Sep 14 08:01:08.149062 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:08.149017    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7" podStartSLOduration=2.149003328 podStartE2EDuration="2.149003328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:01:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:01:08.146377106 +0000 UTC m=+788.775264760" watchObservedRunningTime="2026-09-14 08:01:08.149003328 +0000 UTC m=+788.777891007"
Sep 14 08:01:14.994263 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:14.994227    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"]
Sep 14 08:01:14.994703 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:14.994483    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v" podUID="5438f7ab-ec36-4eaf-a864-0253f9c631b3" containerName="payload-processing" containerID="cri-o://480d666f760faa46f0db30866577512cd0ca0d78db9a8a97ac04fe034b6ee64c" gracePeriod=30
Sep 14 08:01:15.173964 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:15.173929    2563 generic.go:356] "Generic (PLEG): container finished" podID="5438f7ab-ec36-4eaf-a864-0253f9c631b3" containerID="480d666f760faa46f0db30866577512cd0ca0d78db9a8a97ac04fe034b6ee64c" exitCode=0
Sep 14 08:01:15.174105 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:15.174018    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v" event={"ID":"5438f7ab-ec36-4eaf-a864-0253f9c631b3","Type":"ContainerDied","Data":"480d666f760faa46f0db30866577512cd0ca0d78db9a8a97ac04fe034b6ee64c"}
Sep 14 08:01:15.271452 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:15.271424    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"
Sep 14 08:01:15.280124 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:15.280099    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm"]
Sep 14 08:01:15.287163 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:15.287139    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a9d21f4e-qgdtm"]
Sep 14 08:01:15.333407 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:15.333371    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9k9d4\" (UniqueName: \"kubernetes.io/projected/5438f7ab-ec36-4eaf-a864-0253f9c631b3-kube-api-access-9k9d4\") pod \"5438f7ab-ec36-4eaf-a864-0253f9c631b3\" (UID: \"5438f7ab-ec36-4eaf-a864-0253f9c631b3\") "
Sep 14 08:01:15.333563 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:15.333432    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5438f7ab-ec36-4eaf-a864-0253f9c631b3-plugins-config-volume\") pod \"5438f7ab-ec36-4eaf-a864-0253f9c631b3\" (UID: \"5438f7ab-ec36-4eaf-a864-0253f9c631b3\") "
Sep 14 08:01:15.333950 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:15.333923    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5438f7ab-ec36-4eaf-a864-0253f9c631b3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5438f7ab-ec36-4eaf-a864-0253f9c631b3" (UID: "5438f7ab-ec36-4eaf-a864-0253f9c631b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:01:15.336200 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:15.336173    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5438f7ab-ec36-4eaf-a864-0253f9c631b3-kube-api-access-9k9d4" (OuterVolumeSpecName: "kube-api-access-9k9d4") pod "5438f7ab-ec36-4eaf-a864-0253f9c631b3" (UID: "5438f7ab-ec36-4eaf-a864-0253f9c631b3"). InnerVolumeSpecName "kube-api-access-9k9d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:01:15.434373 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:15.434336    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k9d4\" (UniqueName: \"kubernetes.io/projected/5438f7ab-ec36-4eaf-a864-0253f9c631b3-kube-api-access-9k9d4\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:01:15.434373 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:15.434375    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5438f7ab-ec36-4eaf-a864-0253f9c631b3-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:01:15.888949 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:15.888914    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aba049e-68e3-4a5e-a7af-e604cc3e8124" path="/var/lib/kubelet/pods/1aba049e-68e3-4a5e-a7af-e604cc3e8124/volumes"
Sep 14 08:01:16.181444 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:16.181355    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"
Sep 14 08:01:16.181444 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:16.181365    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v" event={"ID":"5438f7ab-ec36-4eaf-a864-0253f9c631b3","Type":"ContainerDied","Data":"05933906d6577e72b0c65f293fca428440b6418d9674ced9619ed55fe7962240"}
Sep 14 08:01:16.181444 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:16.181432    2563 scope.go:117] "RemoveContainer" containerID="480d666f760faa46f0db30866577512cd0ca0d78db9a8a97ac04fe034b6ee64c"
Sep 14 08:01:16.199048 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:16.198977    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"]
Sep 14 08:01:16.205027 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:16.204997    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a9d21f4e-66b86d77f5-d8l5v"]
Sep 14 08:01:17.886977 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:17.886938    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5438f7ab-ec36-4eaf-a864-0253f9c631b3" path="/var/lib/kubelet/pods/5438f7ab-ec36-4eaf-a864-0253f9c631b3/volumes"
Sep 14 08:01:18.127996 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:18.127970    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"
Sep 14 08:01:19.133819 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:19.133792    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"
Sep 14 08:01:22.185399 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.185310    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"]
Sep 14 08:01:22.186045 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.186025    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5438f7ab-ec36-4eaf-a864-0253f9c631b3" containerName="payload-processing"
Sep 14 08:01:22.186045 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.186046    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5438f7ab-ec36-4eaf-a864-0253f9c631b3" containerName="payload-processing"
Sep 14 08:01:22.186124 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.186058    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aba049e-68e3-4a5e-a7af-e604cc3e8124" containerName="revoke-keys"
Sep 14 08:01:22.186124 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.186064    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aba049e-68e3-4a5e-a7af-e604cc3e8124" containerName="revoke-keys"
Sep 14 08:01:22.186186 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.186145    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aba049e-68e3-4a5e-a7af-e604cc3e8124" containerName="revoke-keys"
Sep 14 08:01:22.186186 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.186158    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5438f7ab-ec36-4eaf-a864-0253f9c631b3" containerName="payload-processing"
Sep 14 08:01:22.190481 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.190463    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"
Sep 14 08:01:22.192690 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.192672    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-71665efb\""
Sep 14 08:01:22.214072 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.201113    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"]
Sep 14 08:01:22.306904 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.306873    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w9gs\" (UniqueName: \"kubernetes.io/projected/4294852d-9d5b-481c-8ee6-6d2a7604b46f-kube-api-access-9w9gs\") pod \"payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6\" (UID: \"4294852d-9d5b-481c-8ee6-6d2a7604b46f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"
Sep 14 08:01:22.307099 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.306912    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4294852d-9d5b-481c-8ee6-6d2a7604b46f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6\" (UID: \"4294852d-9d5b-481c-8ee6-6d2a7604b46f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"
Sep 14 08:01:22.408607 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.408565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w9gs\" (UniqueName: \"kubernetes.io/projected/4294852d-9d5b-481c-8ee6-6d2a7604b46f-kube-api-access-9w9gs\") pod \"payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6\" (UID: \"4294852d-9d5b-481c-8ee6-6d2a7604b46f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"
Sep 14 08:01:22.408607 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.408607    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4294852d-9d5b-481c-8ee6-6d2a7604b46f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6\" (UID: \"4294852d-9d5b-481c-8ee6-6d2a7604b46f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"
Sep 14 08:01:22.409348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.409326    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4294852d-9d5b-481c-8ee6-6d2a7604b46f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6\" (UID: \"4294852d-9d5b-481c-8ee6-6d2a7604b46f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"
Sep 14 08:01:22.417711 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.417679    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w9gs\" (UniqueName: \"kubernetes.io/projected/4294852d-9d5b-481c-8ee6-6d2a7604b46f-kube-api-access-9w9gs\") pod \"payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6\" (UID: \"4294852d-9d5b-481c-8ee6-6d2a7604b46f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"
Sep 14 08:01:22.506022 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.505944    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"
Sep 14 08:01:22.783412 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:22.783385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"]
Sep 14 08:01:22.784115 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:01:22.784093    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4294852d_9d5b_481c_8ee6_6d2a7604b46f.slice/crio-dfa01c83b4afcae7b1981f11ad84ebb93caaebefa4756d1b955c4c2e480ef8cf WatchSource:0}: Error finding container dfa01c83b4afcae7b1981f11ad84ebb93caaebefa4756d1b955c4c2e480ef8cf: Status 404 returned error can't find the container with id dfa01c83b4afcae7b1981f11ad84ebb93caaebefa4756d1b955c4c2e480ef8cf
Sep 14 08:01:23.243161 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:23.243125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6" event={"ID":"4294852d-9d5b-481c-8ee6-6d2a7604b46f","Type":"ContainerStarted","Data":"f761ebb35ee95f5f5ed1ec20ef96835310b36ef7a582ab1049e9e046deb0a6bd"}
Sep 14 08:01:23.243664 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:23.243188    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"
Sep 14 08:01:23.243664 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:23.243214    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6" event={"ID":"4294852d-9d5b-481c-8ee6-6d2a7604b46f","Type":"ContainerStarted","Data":"dfa01c83b4afcae7b1981f11ad84ebb93caaebefa4756d1b955c4c2e480ef8cf"}
Sep 14 08:01:23.262736 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:23.262687    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6" podStartSLOduration=1.2626693979999999 podStartE2EDuration="1.262669398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:01:23.259781817 +0000 UTC m=+803.888669468" watchObservedRunningTime="2026-09-14 08:01:23.262669398 +0000 UTC m=+803.891557025"
Sep 14 08:01:34.252279 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:01:34.252249    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"
Sep 14 08:02:01.171878 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.171844    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"]
Sep 14 08:02:01.172412 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.172097    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6" podUID="4294852d-9d5b-481c-8ee6-6d2a7604b46f" containerName="payload-pre-processing" containerID="cri-o://f761ebb35ee95f5f5ed1ec20ef96835310b36ef7a582ab1049e9e046deb0a6bd" gracePeriod=30
Sep 14 08:02:01.423855 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.423782    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"
Sep 14 08:02:01.479512 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.479477    2563 generic.go:356] "Generic (PLEG): container finished" podID="4294852d-9d5b-481c-8ee6-6d2a7604b46f" containerID="f761ebb35ee95f5f5ed1ec20ef96835310b36ef7a582ab1049e9e046deb0a6bd" exitCode=0
Sep 14 08:02:01.480809 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.479628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6" event={"ID":"4294852d-9d5b-481c-8ee6-6d2a7604b46f","Type":"ContainerDied","Data":"f761ebb35ee95f5f5ed1ec20ef96835310b36ef7a582ab1049e9e046deb0a6bd"}
Sep 14 08:02:01.481014 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.479782    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"
Sep 14 08:02:01.482348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.480089    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w9gs\" (UniqueName: \"kubernetes.io/projected/4294852d-9d5b-481c-8ee6-6d2a7604b46f-kube-api-access-9w9gs\") pod \"4294852d-9d5b-481c-8ee6-6d2a7604b46f\" (UID: \"4294852d-9d5b-481c-8ee6-6d2a7604b46f\") "
Sep 14 08:02:01.482348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.481331    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4294852d-9d5b-481c-8ee6-6d2a7604b46f-plugins-config-volume\") pod \"4294852d-9d5b-481c-8ee6-6d2a7604b46f\" (UID: \"4294852d-9d5b-481c-8ee6-6d2a7604b46f\") "
Sep 14 08:02:01.482348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.481739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6" event={"ID":"4294852d-9d5b-481c-8ee6-6d2a7604b46f","Type":"ContainerDied","Data":"dfa01c83b4afcae7b1981f11ad84ebb93caaebefa4756d1b955c4c2e480ef8cf"}
Sep 14 08:02:01.482348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.481784    2563 scope.go:117] "RemoveContainer" containerID="f761ebb35ee95f5f5ed1ec20ef96835310b36ef7a582ab1049e9e046deb0a6bd"
Sep 14 08:02:01.482348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.482299    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4294852d-9d5b-481c-8ee6-6d2a7604b46f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4294852d-9d5b-481c-8ee6-6d2a7604b46f" (UID: "4294852d-9d5b-481c-8ee6-6d2a7604b46f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:02:01.484633 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.484595    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4294852d-9d5b-481c-8ee6-6d2a7604b46f-kube-api-access-9w9gs" (OuterVolumeSpecName: "kube-api-access-9w9gs") pod "4294852d-9d5b-481c-8ee6-6d2a7604b46f" (UID: "4294852d-9d5b-481c-8ee6-6d2a7604b46f"). InnerVolumeSpecName "kube-api-access-9w9gs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:02:01.504078 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.502962    2563 scope.go:117] "RemoveContainer" containerID="f761ebb35ee95f5f5ed1ec20ef96835310b36ef7a582ab1049e9e046deb0a6bd"
Sep 14 08:02:01.505064 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:02:01.504987    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f761ebb35ee95f5f5ed1ec20ef96835310b36ef7a582ab1049e9e046deb0a6bd\": container with ID starting with f761ebb35ee95f5f5ed1ec20ef96835310b36ef7a582ab1049e9e046deb0a6bd not found: ID does not exist" containerID="f761ebb35ee95f5f5ed1ec20ef96835310b36ef7a582ab1049e9e046deb0a6bd"
Sep 14 08:02:01.505064 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.505027    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f761ebb35ee95f5f5ed1ec20ef96835310b36ef7a582ab1049e9e046deb0a6bd"} err="failed to get container status \"f761ebb35ee95f5f5ed1ec20ef96835310b36ef7a582ab1049e9e046deb0a6bd\": rpc error: code = NotFound desc = could not find container \"f761ebb35ee95f5f5ed1ec20ef96835310b36ef7a582ab1049e9e046deb0a6bd\": container with ID starting with f761ebb35ee95f5f5ed1ec20ef96835310b36ef7a582ab1049e9e046deb0a6bd not found: ID does not exist"
Sep 14 08:02:01.582934 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.582901    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w9gs\" (UniqueName: \"kubernetes.io/projected/4294852d-9d5b-481c-8ee6-6d2a7604b46f-kube-api-access-9w9gs\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:02:01.582934 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.582927    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4294852d-9d5b-481c-8ee6-6d2a7604b46f-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:02:01.810034 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.809999    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"]
Sep 14 08:02:01.813348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.813322    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-71665efb-678bd67558-2cpk6"]
Sep 14 08:02:01.886505 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:01.886470    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4294852d-9d5b-481c-8ee6-6d2a7604b46f" path="/var/lib/kubelet/pods/4294852d-9d5b-481c-8ee6-6d2a7604b46f/volumes"
Sep 14 08:02:08.549104 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.549063    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"]
Sep 14 08:02:08.549672 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.549601    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4294852d-9d5b-481c-8ee6-6d2a7604b46f" containerName="payload-pre-processing"
Sep 14 08:02:08.549672 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.549619    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4294852d-9d5b-481c-8ee6-6d2a7604b46f" containerName="payload-pre-processing"
Sep 14 08:02:08.549758 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.549696    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4294852d-9d5b-481c-8ee6-6d2a7604b46f" containerName="payload-pre-processing"
Sep 14 08:02:08.554480 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.554461    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"
Sep 14 08:02:08.557459 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.557437    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-fd181c19\""
Sep 14 08:02:08.557564 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.557547    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-fd181c19-dockercfg-pdq7p\""
Sep 14 08:02:08.563889 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.563866    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"]
Sep 14 08:02:08.651918 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.651893    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/103147ee-cbf8-477a-8a4a-9c326fb59f4e-plugins-config-volume\") pod \"payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6\" (UID: \"103147ee-cbf8-477a-8a4a-9c326fb59f4e\") " pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"
Sep 14 08:02:08.652060 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.652013    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5bkb\" (UniqueName: \"kubernetes.io/projected/103147ee-cbf8-477a-8a4a-9c326fb59f4e-kube-api-access-n5bkb\") pod \"payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6\" (UID: \"103147ee-cbf8-477a-8a4a-9c326fb59f4e\") " pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"
Sep 14 08:02:08.753339 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.753289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5bkb\" (UniqueName: \"kubernetes.io/projected/103147ee-cbf8-477a-8a4a-9c326fb59f4e-kube-api-access-n5bkb\") pod \"payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6\" (UID: \"103147ee-cbf8-477a-8a4a-9c326fb59f4e\") " pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"
Sep 14 08:02:08.753538 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.753368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/103147ee-cbf8-477a-8a4a-9c326fb59f4e-plugins-config-volume\") pod \"payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6\" (UID: \"103147ee-cbf8-477a-8a4a-9c326fb59f4e\") " pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"
Sep 14 08:02:08.754117 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.754096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/103147ee-cbf8-477a-8a4a-9c326fb59f4e-plugins-config-volume\") pod \"payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6\" (UID: \"103147ee-cbf8-477a-8a4a-9c326fb59f4e\") " pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"
Sep 14 08:02:08.762018 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.761994    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5bkb\" (UniqueName: \"kubernetes.io/projected/103147ee-cbf8-477a-8a4a-9c326fb59f4e-kube-api-access-n5bkb\") pod \"payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6\" (UID: \"103147ee-cbf8-477a-8a4a-9c326fb59f4e\") " pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"
Sep 14 08:02:08.866029 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:08.866000    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"
Sep 14 08:02:09.139129 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:09.139096    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"]
Sep 14 08:02:09.140887 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:02:09.140833    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103147ee_cbf8_477a_8a4a_9c326fb59f4e.slice/crio-f937343946c9700a91f5b8fca1b143ac9e30a4438f021072a4da4fdcdec28fb7 WatchSource:0}: Error finding container f937343946c9700a91f5b8fca1b143ac9e30a4438f021072a4da4fdcdec28fb7: Status 404 returned error can't find the container with id f937343946c9700a91f5b8fca1b143ac9e30a4438f021072a4da4fdcdec28fb7
Sep 14 08:02:09.534559 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:09.534458    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6" event={"ID":"103147ee-cbf8-477a-8a4a-9c326fb59f4e","Type":"ContainerStarted","Data":"641e8f140a6fdfb9c72b3797a3e928e9f7dd28c5e006b25a7e19a7344480cbb2"}
Sep 14 08:02:09.534721 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:09.534596    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"
Sep 14 08:02:09.534721 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:09.534627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6" event={"ID":"103147ee-cbf8-477a-8a4a-9c326fb59f4e","Type":"ContainerStarted","Data":"f937343946c9700a91f5b8fca1b143ac9e30a4438f021072a4da4fdcdec28fb7"}
Sep 14 08:02:09.552011 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:09.551966    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6" podStartSLOduration=1.551951986 podStartE2EDuration="1.551951986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:09.550834892 +0000 UTC m=+850.179722539" watchObservedRunningTime="2026-09-14 08:02:09.551951986 +0000 UTC m=+850.180839654"
Sep 14 08:02:20.545025 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:20.544995    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"
Sep 14 08:02:38.255306 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.254314    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"]
Sep 14 08:02:38.255306 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.254571    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6" podUID="103147ee-cbf8-477a-8a4a-9c326fb59f4e" containerName="payload-processing" containerID="cri-o://641e8f140a6fdfb9c72b3797a3e928e9f7dd28c5e006b25a7e19a7344480cbb2" gracePeriod=30
Sep 14 08:02:38.502599 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.502537    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"
Sep 14 08:02:38.518900 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.518877    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5bkb\" (UniqueName: \"kubernetes.io/projected/103147ee-cbf8-477a-8a4a-9c326fb59f4e-kube-api-access-n5bkb\") pod \"103147ee-cbf8-477a-8a4a-9c326fb59f4e\" (UID: \"103147ee-cbf8-477a-8a4a-9c326fb59f4e\") "
Sep 14 08:02:38.519022 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.518949    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/103147ee-cbf8-477a-8a4a-9c326fb59f4e-plugins-config-volume\") pod \"103147ee-cbf8-477a-8a4a-9c326fb59f4e\" (UID: \"103147ee-cbf8-477a-8a4a-9c326fb59f4e\") "
Sep 14 08:02:38.519307 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.519281    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/103147ee-cbf8-477a-8a4a-9c326fb59f4e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "103147ee-cbf8-477a-8a4a-9c326fb59f4e" (UID: "103147ee-cbf8-477a-8a4a-9c326fb59f4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:02:38.520893 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.520874    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/103147ee-cbf8-477a-8a4a-9c326fb59f4e-kube-api-access-n5bkb" (OuterVolumeSpecName: "kube-api-access-n5bkb") pod "103147ee-cbf8-477a-8a4a-9c326fb59f4e" (UID: "103147ee-cbf8-477a-8a4a-9c326fb59f4e"). InnerVolumeSpecName "kube-api-access-n5bkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:02:38.619992 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.619961    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5bkb\" (UniqueName: \"kubernetes.io/projected/103147ee-cbf8-477a-8a4a-9c326fb59f4e-kube-api-access-n5bkb\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:02:38.619992 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.619989    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/103147ee-cbf8-477a-8a4a-9c326fb59f4e-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:02:38.713488 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.713452    2563 generic.go:356] "Generic (PLEG): container finished" podID="103147ee-cbf8-477a-8a4a-9c326fb59f4e" containerID="641e8f140a6fdfb9c72b3797a3e928e9f7dd28c5e006b25a7e19a7344480cbb2" exitCode=0
Sep 14 08:02:38.713670 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.713524    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"
Sep 14 08:02:38.713670 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.713538    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6" event={"ID":"103147ee-cbf8-477a-8a4a-9c326fb59f4e","Type":"ContainerDied","Data":"641e8f140a6fdfb9c72b3797a3e928e9f7dd28c5e006b25a7e19a7344480cbb2"}
Sep 14 08:02:38.713670 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.713623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6" event={"ID":"103147ee-cbf8-477a-8a4a-9c326fb59f4e","Type":"ContainerDied","Data":"f937343946c9700a91f5b8fca1b143ac9e30a4438f021072a4da4fdcdec28fb7"}
Sep 14 08:02:38.713670 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.713661    2563 scope.go:117] "RemoveContainer" containerID="641e8f140a6fdfb9c72b3797a3e928e9f7dd28c5e006b25a7e19a7344480cbb2"
Sep 14 08:02:38.723808 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.723790    2563 scope.go:117] "RemoveContainer" containerID="641e8f140a6fdfb9c72b3797a3e928e9f7dd28c5e006b25a7e19a7344480cbb2"
Sep 14 08:02:38.724085 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:02:38.724066    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"641e8f140a6fdfb9c72b3797a3e928e9f7dd28c5e006b25a7e19a7344480cbb2\": container with ID starting with 641e8f140a6fdfb9c72b3797a3e928e9f7dd28c5e006b25a7e19a7344480cbb2 not found: ID does not exist" containerID="641e8f140a6fdfb9c72b3797a3e928e9f7dd28c5e006b25a7e19a7344480cbb2"
Sep 14 08:02:38.724141 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.724095    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"641e8f140a6fdfb9c72b3797a3e928e9f7dd28c5e006b25a7e19a7344480cbb2"} err="failed to get container status \"641e8f140a6fdfb9c72b3797a3e928e9f7dd28c5e006b25a7e19a7344480cbb2\": rpc error: code = NotFound desc = could not find container \"641e8f140a6fdfb9c72b3797a3e928e9f7dd28c5e006b25a7e19a7344480cbb2\": container with ID starting with 641e8f140a6fdfb9c72b3797a3e928e9f7dd28c5e006b25a7e19a7344480cbb2 not found: ID does not exist"
Sep 14 08:02:38.735607 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.735582    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"]
Sep 14 08:02:38.744234 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:38.744211    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fd181c19-dd7cb96fc-pd9s6"]
Sep 14 08:02:39.887336 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:39.887308    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="103147ee-cbf8-477a-8a4a-9c326fb59f4e" path="/var/lib/kubelet/pods/103147ee-cbf8-477a-8a4a-9c326fb59f4e/volumes"
Sep 14 08:02:45.046906 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.046868    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"]
Sep 14 08:02:45.047533 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.047515    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="103147ee-cbf8-477a-8a4a-9c326fb59f4e" containerName="payload-processing"
Sep 14 08:02:45.047533 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.047534    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="103147ee-cbf8-477a-8a4a-9c326fb59f4e" containerName="payload-processing"
Sep 14 08:02:45.047681 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.047631    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="103147ee-cbf8-477a-8a4a-9c326fb59f4e" containerName="payload-processing"
Sep 14 08:02:45.052359 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.052341    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"
Sep 14 08:02:45.054627 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.054592    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-b931e88a-dockercfg-msr2c\""
Sep 14 08:02:45.054763 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.054628    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-b931e88a\""
Sep 14 08:02:45.059133 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.059113    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"]
Sep 14 08:02:45.083094 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.083071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce471b2b-13ae-4648-bf94-c79a698d7adb-plugins-config-volume\") pod \"payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf\" (UID: \"ce471b2b-13ae-4648-bf94-c79a698d7adb\") " pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"
Sep 14 08:02:45.083094 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.083098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7kq9\" (UniqueName: \"kubernetes.io/projected/ce471b2b-13ae-4648-bf94-c79a698d7adb-kube-api-access-l7kq9\") pod \"payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf\" (UID: \"ce471b2b-13ae-4648-bf94-c79a698d7adb\") " pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"
Sep 14 08:02:45.094635 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.094601    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"]
Sep 14 08:02:45.097923 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.097908    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"
Sep 14 08:02:45.109404 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.109377    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"]
Sep 14 08:02:45.184459 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.184428    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce471b2b-13ae-4648-bf94-c79a698d7adb-plugins-config-volume\") pod \"payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf\" (UID: \"ce471b2b-13ae-4648-bf94-c79a698d7adb\") " pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"
Sep 14 08:02:45.184459 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.184459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7kq9\" (UniqueName: \"kubernetes.io/projected/ce471b2b-13ae-4648-bf94-c79a698d7adb-kube-api-access-l7kq9\") pod \"payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf\" (UID: \"ce471b2b-13ae-4648-bf94-c79a698d7adb\") " pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"
Sep 14 08:02:45.184861 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.184491    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8sjk\" (UniqueName: \"kubernetes.io/projected/d4f5556d-5df3-411e-859a-f77c7f603f13-kube-api-access-x8sjk\") pod \"payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4\" (UID: \"d4f5556d-5df3-411e-859a-f77c7f603f13\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"
Sep 14 08:02:45.184861 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.184521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f5556d-5df3-411e-859a-f77c7f603f13-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4\" (UID: \"d4f5556d-5df3-411e-859a-f77c7f603f13\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"
Sep 14 08:02:45.185195 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.185172    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce471b2b-13ae-4648-bf94-c79a698d7adb-plugins-config-volume\") pod \"payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf\" (UID: \"ce471b2b-13ae-4648-bf94-c79a698d7adb\") " pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"
Sep 14 08:02:45.194269 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.194241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7kq9\" (UniqueName: \"kubernetes.io/projected/ce471b2b-13ae-4648-bf94-c79a698d7adb-kube-api-access-l7kq9\") pod \"payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf\" (UID: \"ce471b2b-13ae-4648-bf94-c79a698d7adb\") " pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"
Sep 14 08:02:45.285610 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.285573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8sjk\" (UniqueName: \"kubernetes.io/projected/d4f5556d-5df3-411e-859a-f77c7f603f13-kube-api-access-x8sjk\") pod \"payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4\" (UID: \"d4f5556d-5df3-411e-859a-f77c7f603f13\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"
Sep 14 08:02:45.285794 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.285619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f5556d-5df3-411e-859a-f77c7f603f13-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4\" (UID: \"d4f5556d-5df3-411e-859a-f77c7f603f13\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"
Sep 14 08:02:45.286300 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.286280    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f5556d-5df3-411e-859a-f77c7f603f13-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4\" (UID: \"d4f5556d-5df3-411e-859a-f77c7f603f13\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"
Sep 14 08:02:45.296512 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.296479    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8sjk\" (UniqueName: \"kubernetes.io/projected/d4f5556d-5df3-411e-859a-f77c7f603f13-kube-api-access-x8sjk\") pod \"payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4\" (UID: \"d4f5556d-5df3-411e-859a-f77c7f603f13\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"
Sep 14 08:02:45.363768 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.363744    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"
Sep 14 08:02:45.408218 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.408188    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"
Sep 14 08:02:45.637613 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.637587    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"]
Sep 14 08:02:45.638332 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:02:45.638296    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce471b2b_13ae_4648_bf94_c79a698d7adb.slice/crio-f738d6b4ab474a0766c10c6214f015b4e0d5b2176618ba697c162908f2b98597 WatchSource:0}: Error finding container f738d6b4ab474a0766c10c6214f015b4e0d5b2176618ba697c162908f2b98597: Status 404 returned error can't find the container with id f738d6b4ab474a0766c10c6214f015b4e0d5b2176618ba697c162908f2b98597
Sep 14 08:02:45.706006 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.705980    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"]
Sep 14 08:02:45.706544 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:02:45.706520    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f5556d_5df3_411e_859a_f77c7f603f13.slice/crio-bb581950ffb936064a2da8076eb1ce2a3b8fce51e563ae5b8b21afe9275c75d5 WatchSource:0}: Error finding container bb581950ffb936064a2da8076eb1ce2a3b8fce51e563ae5b8b21afe9275c75d5: Status 404 returned error can't find the container with id bb581950ffb936064a2da8076eb1ce2a3b8fce51e563ae5b8b21afe9275c75d5
Sep 14 08:02:45.767600 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.767574    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf" event={"ID":"ce471b2b-13ae-4648-bf94-c79a698d7adb","Type":"ContainerStarted","Data":"88dac44b662190507e36402e28371ef13fc14e928616f8acda816550541ce66a"}
Sep 14 08:02:45.767748 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.767622    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf" event={"ID":"ce471b2b-13ae-4648-bf94-c79a698d7adb","Type":"ContainerStarted","Data":"f738d6b4ab474a0766c10c6214f015b4e0d5b2176618ba697c162908f2b98597"}
Sep 14 08:02:45.767748 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.767659    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"
Sep 14 08:02:45.769503 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.769481    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4" event={"ID":"d4f5556d-5df3-411e-859a-f77c7f603f13","Type":"ContainerStarted","Data":"bb581950ffb936064a2da8076eb1ce2a3b8fce51e563ae5b8b21afe9275c75d5"}
Sep 14 08:02:45.784669 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:45.784607    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf" podStartSLOduration=0.784591945 podStartE2EDuration="784.591945ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:45.78278001 +0000 UTC m=+886.411667657" watchObservedRunningTime="2026-09-14 08:02:45.784591945 +0000 UTC m=+886.413479569"
Sep 14 08:02:46.777779 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:46.777743    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4" event={"ID":"d4f5556d-5df3-411e-859a-f77c7f603f13","Type":"ContainerStarted","Data":"9468bd6fe45c739f9b38e2831377e5e17348cec036c7f097e3b059c477cb4a10"}
Sep 14 08:02:46.778240 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:46.777985    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"
Sep 14 08:02:46.798435 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:46.798382    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4" podStartSLOduration=1.798367502 podStartE2EDuration="1.798367502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:46.795168936 +0000 UTC m=+887.424056580" watchObservedRunningTime="2026-09-14 08:02:46.798367502 +0000 UTC m=+887.427255126"
Sep 14 08:02:56.780134 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:56.780105    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"
Sep 14 08:02:57.786225 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:57.786198    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"
Sep 14 08:02:59.839142 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:59.839111    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 08:02:59.840386 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:02:59.840365    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 08:03:16.034741 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.034703    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"]
Sep 14 08:03:16.037254 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.034959    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf" podUID="ce471b2b-13ae-4648-bf94-c79a698d7adb" containerName="payload-processing" containerID="cri-o://88dac44b662190507e36402e28371ef13fc14e928616f8acda816550541ce66a" gracePeriod=30
Sep 14 08:03:16.049478 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.049449    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"]
Sep 14 08:03:16.049738 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.049691    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4" podUID="d4f5556d-5df3-411e-859a-f77c7f603f13" containerName="payload-pre-processing" containerID="cri-o://9468bd6fe45c739f9b38e2831377e5e17348cec036c7f097e3b059c477cb4a10" gracePeriod=30
Sep 14 08:03:16.363312 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.363288    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"
Sep 14 08:03:16.366684 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.366663    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"
Sep 14 08:03:16.388757 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.388733    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8sjk\" (UniqueName: \"kubernetes.io/projected/d4f5556d-5df3-411e-859a-f77c7f603f13-kube-api-access-x8sjk\") pod \"d4f5556d-5df3-411e-859a-f77c7f603f13\" (UID: \"d4f5556d-5df3-411e-859a-f77c7f603f13\") "
Sep 14 08:03:16.388864 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.388809    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f5556d-5df3-411e-859a-f77c7f603f13-plugins-config-volume\") pod \"d4f5556d-5df3-411e-859a-f77c7f603f13\" (UID: \"d4f5556d-5df3-411e-859a-f77c7f603f13\") "
Sep 14 08:03:16.388947 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.388929    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce471b2b-13ae-4648-bf94-c79a698d7adb-plugins-config-volume\") pod \"ce471b2b-13ae-4648-bf94-c79a698d7adb\" (UID: \"ce471b2b-13ae-4648-bf94-c79a698d7adb\") "
Sep 14 08:03:16.389089 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.389071    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7kq9\" (UniqueName: \"kubernetes.io/projected/ce471b2b-13ae-4648-bf94-c79a698d7adb-kube-api-access-l7kq9\") pod \"ce471b2b-13ae-4648-bf94-c79a698d7adb\" (UID: \"ce471b2b-13ae-4648-bf94-c79a698d7adb\") "
Sep 14 08:03:16.389254 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.389232    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4f5556d-5df3-411e-859a-f77c7f603f13-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4f5556d-5df3-411e-859a-f77c7f603f13" (UID: "d4f5556d-5df3-411e-859a-f77c7f603f13"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:16.389357 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.389333    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce471b2b-13ae-4648-bf94-c79a698d7adb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce471b2b-13ae-4648-bf94-c79a698d7adb" (UID: "ce471b2b-13ae-4648-bf94-c79a698d7adb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:16.389517 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.389495    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f5556d-5df3-411e-859a-f77c7f603f13-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:03:16.389869 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.389519    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce471b2b-13ae-4648-bf94-c79a698d7adb-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:03:16.391300 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.391274    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f5556d-5df3-411e-859a-f77c7f603f13-kube-api-access-x8sjk" (OuterVolumeSpecName: "kube-api-access-x8sjk") pod "d4f5556d-5df3-411e-859a-f77c7f603f13" (UID: "d4f5556d-5df3-411e-859a-f77c7f603f13"). InnerVolumeSpecName "kube-api-access-x8sjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:16.391457 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.391436    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce471b2b-13ae-4648-bf94-c79a698d7adb-kube-api-access-l7kq9" (OuterVolumeSpecName: "kube-api-access-l7kq9") pod "ce471b2b-13ae-4648-bf94-c79a698d7adb" (UID: "ce471b2b-13ae-4648-bf94-c79a698d7adb"). InnerVolumeSpecName "kube-api-access-l7kq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:16.490177 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.490143    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7kq9\" (UniqueName: \"kubernetes.io/projected/ce471b2b-13ae-4648-bf94-c79a698d7adb-kube-api-access-l7kq9\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:03:16.490177 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.490171    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8sjk\" (UniqueName: \"kubernetes.io/projected/d4f5556d-5df3-411e-859a-f77c7f603f13-kube-api-access-x8sjk\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:03:16.969031 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.968998    2563 generic.go:356] "Generic (PLEG): container finished" podID="ce471b2b-13ae-4648-bf94-c79a698d7adb" containerID="88dac44b662190507e36402e28371ef13fc14e928616f8acda816550541ce66a" exitCode=0
Sep 14 08:03:16.969236 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.969043    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"
Sep 14 08:03:16.969236 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.969087    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf" event={"ID":"ce471b2b-13ae-4648-bf94-c79a698d7adb","Type":"ContainerDied","Data":"88dac44b662190507e36402e28371ef13fc14e928616f8acda816550541ce66a"}
Sep 14 08:03:16.969236 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.969144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf" event={"ID":"ce471b2b-13ae-4648-bf94-c79a698d7adb","Type":"ContainerDied","Data":"f738d6b4ab474a0766c10c6214f015b4e0d5b2176618ba697c162908f2b98597"}
Sep 14 08:03:16.969236 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.969167    2563 scope.go:117] "RemoveContainer" containerID="88dac44b662190507e36402e28371ef13fc14e928616f8acda816550541ce66a"
Sep 14 08:03:16.970589 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.970564    2563 generic.go:356] "Generic (PLEG): container finished" podID="d4f5556d-5df3-411e-859a-f77c7f603f13" containerID="9468bd6fe45c739f9b38e2831377e5e17348cec036c7f097e3b059c477cb4a10" exitCode=0
Sep 14 08:03:16.970716 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.970615    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"
Sep 14 08:03:16.970772 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.970626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4" event={"ID":"d4f5556d-5df3-411e-859a-f77c7f603f13","Type":"ContainerDied","Data":"9468bd6fe45c739f9b38e2831377e5e17348cec036c7f097e3b059c477cb4a10"}
Sep 14 08:03:16.970811 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.970795    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4" event={"ID":"d4f5556d-5df3-411e-859a-f77c7f603f13","Type":"ContainerDied","Data":"bb581950ffb936064a2da8076eb1ce2a3b8fce51e563ae5b8b21afe9275c75d5"}
Sep 14 08:03:16.981848 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.981829    2563 scope.go:117] "RemoveContainer" containerID="88dac44b662190507e36402e28371ef13fc14e928616f8acda816550541ce66a"
Sep 14 08:03:16.982149 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:03:16.982124    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88dac44b662190507e36402e28371ef13fc14e928616f8acda816550541ce66a\": container with ID starting with 88dac44b662190507e36402e28371ef13fc14e928616f8acda816550541ce66a not found: ID does not exist" containerID="88dac44b662190507e36402e28371ef13fc14e928616f8acda816550541ce66a"
Sep 14 08:03:16.982250 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.982160    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88dac44b662190507e36402e28371ef13fc14e928616f8acda816550541ce66a"} err="failed to get container status \"88dac44b662190507e36402e28371ef13fc14e928616f8acda816550541ce66a\": rpc error: code = NotFound desc = could not find container \"88dac44b662190507e36402e28371ef13fc14e928616f8acda816550541ce66a\": container with ID starting with 88dac44b662190507e36402e28371ef13fc14e928616f8acda816550541ce66a not found: ID does not exist"
Sep 14 08:03:16.982250 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.982184    2563 scope.go:117] "RemoveContainer" containerID="9468bd6fe45c739f9b38e2831377e5e17348cec036c7f097e3b059c477cb4a10"
Sep 14 08:03:16.991144 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.991124    2563 scope.go:117] "RemoveContainer" containerID="9468bd6fe45c739f9b38e2831377e5e17348cec036c7f097e3b059c477cb4a10"
Sep 14 08:03:16.991395 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:03:16.991374    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9468bd6fe45c739f9b38e2831377e5e17348cec036c7f097e3b059c477cb4a10\": container with ID starting with 9468bd6fe45c739f9b38e2831377e5e17348cec036c7f097e3b059c477cb4a10 not found: ID does not exist" containerID="9468bd6fe45c739f9b38e2831377e5e17348cec036c7f097e3b059c477cb4a10"
Sep 14 08:03:16.991455 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:16.991405    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9468bd6fe45c739f9b38e2831377e5e17348cec036c7f097e3b059c477cb4a10"} err="failed to get container status \"9468bd6fe45c739f9b38e2831377e5e17348cec036c7f097e3b059c477cb4a10\": rpc error: code = NotFound desc = could not find container \"9468bd6fe45c739f9b38e2831377e5e17348cec036c7f097e3b059c477cb4a10\": container with ID starting with 9468bd6fe45c739f9b38e2831377e5e17348cec036c7f097e3b059c477cb4a10 not found: ID does not exist"
Sep 14 08:03:17.006525 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:17.006491    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"]
Sep 14 08:03:17.011796 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:17.011772    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b931e88a-d7d7dbf65-gdrdf"]
Sep 14 08:03:17.037070 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:17.037048    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"]
Sep 14 08:03:17.041490 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:17.041466    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b931e88a-f79f58f8-985l4"]
Sep 14 08:03:17.886928 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:17.886898    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce471b2b-13ae-4648-bf94-c79a698d7adb" path="/var/lib/kubelet/pods/ce471b2b-13ae-4648-bf94-c79a698d7adb/volumes"
Sep 14 08:03:17.887274 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:17.887261    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4f5556d-5df3-411e-859a-f77c7f603f13" path="/var/lib/kubelet/pods/d4f5556d-5df3-411e-859a-f77c7f603f13/volumes"
Sep 14 08:03:53.535076 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.535037    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"]
Sep 14 08:03:53.535715 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.535274    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw" podUID="9d8a82c9-6642-4ee6-bcb4-dc08071b8207" containerName="payload-processing" containerID="cri-o://2f0309a9fed2e63d9120eebad6d929d01abe458a4121cd74aee83f5ae78fb393" gracePeriod=30
Sep 14 08:03:53.542520 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.542492    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"]
Sep 14 08:03:53.542764 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.542717    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp" podUID="8e49d370-ad30-4bc5-a780-e77432baa704" containerName="payload-pre-processing" containerID="cri-o://0e2d5041ea5b110f30080e99b216eb480e44b04020d71994cf803053aeb4adbc" gracePeriod=30
Sep 14 08:03:53.835357 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.835333    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"
Sep 14 08:03:53.838413 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.838387    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"
Sep 14 08:03:53.856777 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.856745    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e49d370-ad30-4bc5-a780-e77432baa704-plugins-config-volume\") pod \"8e49d370-ad30-4bc5-a780-e77432baa704\" (UID: \"8e49d370-ad30-4bc5-a780-e77432baa704\") "
Sep 14 08:03:53.856777 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.856787    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8a82c9-6642-4ee6-bcb4-dc08071b8207-plugins-config-volume\") pod \"9d8a82c9-6642-4ee6-bcb4-dc08071b8207\" (UID: \"9d8a82c9-6642-4ee6-bcb4-dc08071b8207\") "
Sep 14 08:03:53.857038 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.856841    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfgkq\" (UniqueName: \"kubernetes.io/projected/8e49d370-ad30-4bc5-a780-e77432baa704-kube-api-access-qfgkq\") pod \"8e49d370-ad30-4bc5-a780-e77432baa704\" (UID: \"8e49d370-ad30-4bc5-a780-e77432baa704\") "
Sep 14 08:03:53.857038 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.856999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfwg6\" (UniqueName: \"kubernetes.io/projected/9d8a82c9-6642-4ee6-bcb4-dc08071b8207-kube-api-access-zfwg6\") pod \"9d8a82c9-6642-4ee6-bcb4-dc08071b8207\" (UID: \"9d8a82c9-6642-4ee6-bcb4-dc08071b8207\") "
Sep 14 08:03:53.857179 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.857154    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e49d370-ad30-4bc5-a780-e77432baa704-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e49d370-ad30-4bc5-a780-e77432baa704" (UID: "8e49d370-ad30-4bc5-a780-e77432baa704"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:53.857242 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.857221    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d8a82c9-6642-4ee6-bcb4-dc08071b8207-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9d8a82c9-6642-4ee6-bcb4-dc08071b8207" (UID: "9d8a82c9-6642-4ee6-bcb4-dc08071b8207"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:53.857387 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.857363    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e49d370-ad30-4bc5-a780-e77432baa704-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:03:53.857454 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.857386    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d8a82c9-6642-4ee6-bcb4-dc08071b8207-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:03:53.860000 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.859415    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e49d370-ad30-4bc5-a780-e77432baa704-kube-api-access-qfgkq" (OuterVolumeSpecName: "kube-api-access-qfgkq") pod "8e49d370-ad30-4bc5-a780-e77432baa704" (UID: "8e49d370-ad30-4bc5-a780-e77432baa704"). InnerVolumeSpecName "kube-api-access-qfgkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:53.860000 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.859990    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d8a82c9-6642-4ee6-bcb4-dc08071b8207-kube-api-access-zfwg6" (OuterVolumeSpecName: "kube-api-access-zfwg6") pod "9d8a82c9-6642-4ee6-bcb4-dc08071b8207" (UID: "9d8a82c9-6642-4ee6-bcb4-dc08071b8207"). InnerVolumeSpecName "kube-api-access-zfwg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:53.958536 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.958499    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfgkq\" (UniqueName: \"kubernetes.io/projected/8e49d370-ad30-4bc5-a780-e77432baa704-kube-api-access-qfgkq\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:03:53.958536 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:53.958533    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfwg6\" (UniqueName: \"kubernetes.io/projected/9d8a82c9-6642-4ee6-bcb4-dc08071b8207-kube-api-access-zfwg6\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:03:54.200506 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.200474    2563 generic.go:356] "Generic (PLEG): container finished" podID="8e49d370-ad30-4bc5-a780-e77432baa704" containerID="0e2d5041ea5b110f30080e99b216eb480e44b04020d71994cf803053aeb4adbc" exitCode=0
Sep 14 08:03:54.200733 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.200516    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"
Sep 14 08:03:54.200733 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.200557    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp" event={"ID":"8e49d370-ad30-4bc5-a780-e77432baa704","Type":"ContainerDied","Data":"0e2d5041ea5b110f30080e99b216eb480e44b04020d71994cf803053aeb4adbc"}
Sep 14 08:03:54.200733 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.200607    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp" event={"ID":"8e49d370-ad30-4bc5-a780-e77432baa704","Type":"ContainerDied","Data":"012e57a00a4aa509138be2893e017e4d3f29726a7166472ef55e7d2f503ec704"}
Sep 14 08:03:54.200733 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.200664    2563 scope.go:117] "RemoveContainer" containerID="0e2d5041ea5b110f30080e99b216eb480e44b04020d71994cf803053aeb4adbc"
Sep 14 08:03:54.202123 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.202094    2563 generic.go:356] "Generic (PLEG): container finished" podID="9d8a82c9-6642-4ee6-bcb4-dc08071b8207" containerID="2f0309a9fed2e63d9120eebad6d929d01abe458a4121cd74aee83f5ae78fb393" exitCode=0
Sep 14 08:03:54.202285 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.202126    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw" event={"ID":"9d8a82c9-6642-4ee6-bcb4-dc08071b8207","Type":"ContainerDied","Data":"2f0309a9fed2e63d9120eebad6d929d01abe458a4121cd74aee83f5ae78fb393"}
Sep 14 08:03:54.202285 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.202138    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"
Sep 14 08:03:54.202285 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.202177    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw" event={"ID":"9d8a82c9-6642-4ee6-bcb4-dc08071b8207","Type":"ContainerDied","Data":"eb85e59293b756678b90e11ae0b23cd1f87e11b5486ca13e4107d9b3c6be1b1c"}
Sep 14 08:03:54.210067 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.210052    2563 scope.go:117] "RemoveContainer" containerID="0e2d5041ea5b110f30080e99b216eb480e44b04020d71994cf803053aeb4adbc"
Sep 14 08:03:54.210317 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:03:54.210300    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e2d5041ea5b110f30080e99b216eb480e44b04020d71994cf803053aeb4adbc\": container with ID starting with 0e2d5041ea5b110f30080e99b216eb480e44b04020d71994cf803053aeb4adbc not found: ID does not exist" containerID="0e2d5041ea5b110f30080e99b216eb480e44b04020d71994cf803053aeb4adbc"
Sep 14 08:03:54.210396 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.210329    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e2d5041ea5b110f30080e99b216eb480e44b04020d71994cf803053aeb4adbc"} err="failed to get container status \"0e2d5041ea5b110f30080e99b216eb480e44b04020d71994cf803053aeb4adbc\": rpc error: code = NotFound desc = could not find container \"0e2d5041ea5b110f30080e99b216eb480e44b04020d71994cf803053aeb4adbc\": container with ID starting with 0e2d5041ea5b110f30080e99b216eb480e44b04020d71994cf803053aeb4adbc not found: ID does not exist"
Sep 14 08:03:54.210396 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.210351    2563 scope.go:117] "RemoveContainer" containerID="2f0309a9fed2e63d9120eebad6d929d01abe458a4121cd74aee83f5ae78fb393"
Sep 14 08:03:54.218504 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.218485    2563 scope.go:117] "RemoveContainer" containerID="2f0309a9fed2e63d9120eebad6d929d01abe458a4121cd74aee83f5ae78fb393"
Sep 14 08:03:54.218792 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:03:54.218773    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f0309a9fed2e63d9120eebad6d929d01abe458a4121cd74aee83f5ae78fb393\": container with ID starting with 2f0309a9fed2e63d9120eebad6d929d01abe458a4121cd74aee83f5ae78fb393 not found: ID does not exist" containerID="2f0309a9fed2e63d9120eebad6d929d01abe458a4121cd74aee83f5ae78fb393"
Sep 14 08:03:54.218857 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.218803    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f0309a9fed2e63d9120eebad6d929d01abe458a4121cd74aee83f5ae78fb393"} err="failed to get container status \"2f0309a9fed2e63d9120eebad6d929d01abe458a4121cd74aee83f5ae78fb393\": rpc error: code = NotFound desc = could not find container \"2f0309a9fed2e63d9120eebad6d929d01abe458a4121cd74aee83f5ae78fb393\": container with ID starting with 2f0309a9fed2e63d9120eebad6d929d01abe458a4121cd74aee83f5ae78fb393 not found: ID does not exist"
Sep 14 08:03:54.230489 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.230467    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"]
Sep 14 08:03:54.236866 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.236845    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-22b517-64c7b95d4b-wxsrp"]
Sep 14 08:03:54.261120 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.261088    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"]
Sep 14 08:03:54.268060 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:54.268016    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-22b517-79f754b75-brgfw"]
Sep 14 08:03:55.370271 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.370231    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn"]
Sep 14 08:03:55.370887 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.370868    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d8a82c9-6642-4ee6-bcb4-dc08071b8207" containerName="payload-processing"
Sep 14 08:03:55.370948 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.370889    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d8a82c9-6642-4ee6-bcb4-dc08071b8207" containerName="payload-processing"
Sep 14 08:03:55.370948 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.370899    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e49d370-ad30-4bc5-a780-e77432baa704" containerName="payload-pre-processing"
Sep 14 08:03:55.370948 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.370904    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e49d370-ad30-4bc5-a780-e77432baa704" containerName="payload-pre-processing"
Sep 14 08:03:55.370948 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.370911    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f5556d-5df3-411e-859a-f77c7f603f13" containerName="payload-pre-processing"
Sep 14 08:03:55.370948 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.370918    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f5556d-5df3-411e-859a-f77c7f603f13" containerName="payload-pre-processing"
Sep 14 08:03:55.370948 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.370928    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce471b2b-13ae-4648-bf94-c79a698d7adb" containerName="payload-processing"
Sep 14 08:03:55.370948 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.370934    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce471b2b-13ae-4648-bf94-c79a698d7adb" containerName="payload-processing"
Sep 14 08:03:55.371164 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.371009    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d8a82c9-6642-4ee6-bcb4-dc08071b8207" containerName="payload-processing"
Sep 14 08:03:55.371164 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.371016    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce471b2b-13ae-4648-bf94-c79a698d7adb" containerName="payload-processing"
Sep 14 08:03:55.371164 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.371023    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e49d370-ad30-4bc5-a780-e77432baa704" containerName="payload-pre-processing"
Sep 14 08:03:55.371164 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.371030    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4f5556d-5df3-411e-859a-f77c7f603f13" containerName="payload-pre-processing"
Sep 14 08:03:55.375448 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.375428    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn"
Sep 14 08:03:55.377829 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.377811    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:03:55.387457 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.387433    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn"]
Sep 14 08:03:55.472032 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.471990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d0ade40-04c9-4c1b-be18-ef543d09911b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn\" (UID: \"0d0ade40-04c9-4c1b-be18-ef543d09911b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn"
Sep 14 08:03:55.573498 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.573467    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d0ade40-04c9-4c1b-be18-ef543d09911b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn\" (UID: \"0d0ade40-04c9-4c1b-be18-ef543d09911b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn"
Sep 14 08:03:55.574244 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.574219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d0ade40-04c9-4c1b-be18-ef543d09911b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn\" (UID: \"0d0ade40-04c9-4c1b-be18-ef543d09911b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn"
Sep 14 08:03:55.687009 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.686925    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn"
Sep 14 08:03:55.886602 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.886568    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e49d370-ad30-4bc5-a780-e77432baa704" path="/var/lib/kubelet/pods/8e49d370-ad30-4bc5-a780-e77432baa704/volumes"
Sep 14 08:03:55.886978 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.886963    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d8a82c9-6642-4ee6-bcb4-dc08071b8207" path="/var/lib/kubelet/pods/9d8a82c9-6642-4ee6-bcb4-dc08071b8207/volumes"
Sep 14 08:03:55.960294 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:55.960267    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn"]
Sep 14 08:03:55.961790 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:03:55.961762    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d0ade40_04c9_4c1b_be18_ef543d09911b.slice/crio-e839adac28ec7e8aa403e9ae6cd7b1a50fb12ea4f6b39d6dd675826bbeddcc87 WatchSource:0}: Error finding container e839adac28ec7e8aa403e9ae6cd7b1a50fb12ea4f6b39d6dd675826bbeddcc87: Status 404 returned error can't find the container with id e839adac28ec7e8aa403e9ae6cd7b1a50fb12ea4f6b39d6dd675826bbeddcc87
Sep 14 08:03:56.217261 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:56.217177    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn" event={"ID":"0d0ade40-04c9-4c1b-be18-ef543d09911b","Type":"ContainerStarted","Data":"e839adac28ec7e8aa403e9ae6cd7b1a50fb12ea4f6b39d6dd675826bbeddcc87"}
Sep 14 08:03:57.226064 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:57.225970    2563 generic.go:356] "Generic (PLEG): container finished" podID="0d0ade40-04c9-4c1b-be18-ef543d09911b" containerID="081753d80042a1869e7175b1add261503ba6db0c8a510c2ce5f176af71a88c79" exitCode=0
Sep 14 08:03:57.226064 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:57.226048    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn" event={"ID":"0d0ade40-04c9-4c1b-be18-ef543d09911b","Type":"ContainerDied","Data":"081753d80042a1869e7175b1add261503ba6db0c8a510c2ce5f176af71a88c79"}
Sep 14 08:03:58.373982 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:58.373956    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn"
Sep 14 08:03:58.401231 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:58.401200    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d0ade40-04c9-4c1b-be18-ef543d09911b-maas-api-service-ca\") pod \"0d0ade40-04c9-4c1b-be18-ef543d09911b\" (UID: \"0d0ade40-04c9-4c1b-be18-ef543d09911b\") "
Sep 14 08:03:58.401557 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:58.401531    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d0ade40-04c9-4c1b-be18-ef543d09911b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0d0ade40-04c9-4c1b-be18-ef543d09911b" (UID: "0d0ade40-04c9-4c1b-be18-ef543d09911b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:58.502553 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:58.502475    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d0ade40-04c9-4c1b-be18-ef543d09911b-maas-api-service-ca\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:03:59.238434 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:59.238402    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn"
Sep 14 08:03:59.238608 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:59.238437    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn" event={"ID":"0d0ade40-04c9-4c1b-be18-ef543d09911b","Type":"ContainerDied","Data":"e839adac28ec7e8aa403e9ae6cd7b1a50fb12ea4f6b39d6dd675826bbeddcc87"}
Sep 14 08:03:59.238608 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:03:59.238481    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e839adac28ec7e8aa403e9ae6cd7b1a50fb12ea4f6b39d6dd675826bbeddcc87"
Sep 14 08:04:06.086208 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.085910    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"]
Sep 14 08:04:06.086719 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.086215    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9" podUID="6c78d8a3-2056-488e-a72e-f05a727c54d7" containerName="payload-processing" containerID="cri-o://1c1d7f27465aa3e190d415ed8c4988f21b15000a7416b9d07db411cff97fdb8d" gracePeriod=30
Sep 14 08:04:06.096330 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.096078    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"]
Sep 14 08:04:06.096837 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.096775    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h" podUID="78e9b78a-6eeb-4c01-a01c-beb29edf387b" containerName="payload-pre-processing" containerID="cri-o://9a70f615b878e4be5935f9fa88713b007e7b9ee1f8948001dce968a2450b9c85" gracePeriod=30
Sep 14 08:04:06.286092 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.285885    2563 generic.go:356] "Generic (PLEG): container finished" podID="78e9b78a-6eeb-4c01-a01c-beb29edf387b" containerID="9a70f615b878e4be5935f9fa88713b007e7b9ee1f8948001dce968a2450b9c85" exitCode=0
Sep 14 08:04:06.286092 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.286031    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h" event={"ID":"78e9b78a-6eeb-4c01-a01c-beb29edf387b","Type":"ContainerDied","Data":"9a70f615b878e4be5935f9fa88713b007e7b9ee1f8948001dce968a2450b9c85"}
Sep 14 08:04:06.288182 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.288140    2563 generic.go:356] "Generic (PLEG): container finished" podID="6c78d8a3-2056-488e-a72e-f05a727c54d7" containerID="1c1d7f27465aa3e190d415ed8c4988f21b15000a7416b9d07db411cff97fdb8d" exitCode=0
Sep 14 08:04:06.288308 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.288236    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9" event={"ID":"6c78d8a3-2056-488e-a72e-f05a727c54d7","Type":"ContainerDied","Data":"1c1d7f27465aa3e190d415ed8c4988f21b15000a7416b9d07db411cff97fdb8d"}
Sep 14 08:04:06.314103 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.314070    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn"]
Sep 14 08:04:06.317980 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.317950    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-814672-qlmxn"]
Sep 14 08:04:06.363928 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.363903    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"
Sep 14 08:04:06.390317 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.390291    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"
Sep 14 08:04:06.473568 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.473536    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhj6w\" (UniqueName: \"kubernetes.io/projected/6c78d8a3-2056-488e-a72e-f05a727c54d7-kube-api-access-xhj6w\") pod \"6c78d8a3-2056-488e-a72e-f05a727c54d7\" (UID: \"6c78d8a3-2056-488e-a72e-f05a727c54d7\") "
Sep 14 08:04:06.473754 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.473577    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xncrw\" (UniqueName: \"kubernetes.io/projected/78e9b78a-6eeb-4c01-a01c-beb29edf387b-kube-api-access-xncrw\") pod \"78e9b78a-6eeb-4c01-a01c-beb29edf387b\" (UID: \"78e9b78a-6eeb-4c01-a01c-beb29edf387b\") "
Sep 14 08:04:06.473754 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.473634    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78e9b78a-6eeb-4c01-a01c-beb29edf387b-plugins-config-volume\") pod \"78e9b78a-6eeb-4c01-a01c-beb29edf387b\" (UID: \"78e9b78a-6eeb-4c01-a01c-beb29edf387b\") "
Sep 14 08:04:06.473754 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.473675    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c78d8a3-2056-488e-a72e-f05a727c54d7-plugins-config-volume\") pod \"6c78d8a3-2056-488e-a72e-f05a727c54d7\" (UID: \"6c78d8a3-2056-488e-a72e-f05a727c54d7\") "
Sep 14 08:04:06.474074 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.474047    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78e9b78a-6eeb-4c01-a01c-beb29edf387b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "78e9b78a-6eeb-4c01-a01c-beb29edf387b" (UID: "78e9b78a-6eeb-4c01-a01c-beb29edf387b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:06.474110 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.474066    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c78d8a3-2056-488e-a72e-f05a727c54d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6c78d8a3-2056-488e-a72e-f05a727c54d7" (UID: "6c78d8a3-2056-488e-a72e-f05a727c54d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:06.475730 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.475706    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c78d8a3-2056-488e-a72e-f05a727c54d7-kube-api-access-xhj6w" (OuterVolumeSpecName: "kube-api-access-xhj6w") pod "6c78d8a3-2056-488e-a72e-f05a727c54d7" (UID: "6c78d8a3-2056-488e-a72e-f05a727c54d7"). InnerVolumeSpecName "kube-api-access-xhj6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:06.475730 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.475720    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78e9b78a-6eeb-4c01-a01c-beb29edf387b-kube-api-access-xncrw" (OuterVolumeSpecName: "kube-api-access-xncrw") pod "78e9b78a-6eeb-4c01-a01c-beb29edf387b" (UID: "78e9b78a-6eeb-4c01-a01c-beb29edf387b"). InnerVolumeSpecName "kube-api-access-xncrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:06.575006 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.574971    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78e9b78a-6eeb-4c01-a01c-beb29edf387b-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:04:06.575006 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.575001    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c78d8a3-2056-488e-a72e-f05a727c54d7-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:04:06.575006 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.575011    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhj6w\" (UniqueName: \"kubernetes.io/projected/6c78d8a3-2056-488e-a72e-f05a727c54d7-kube-api-access-xhj6w\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:04:06.575237 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:06.575022    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xncrw\" (UniqueName: \"kubernetes.io/projected/78e9b78a-6eeb-4c01-a01c-beb29edf387b-kube-api-access-xncrw\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:04:07.295688 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:07.295654    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"
Sep 14 08:04:07.295688 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:07.295673    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h" event={"ID":"78e9b78a-6eeb-4c01-a01c-beb29edf387b","Type":"ContainerDied","Data":"99e710fd54ef686d1674a21f09533e02a7f3d8f2cd987bbc8499e757653148a2"}
Sep 14 08:04:07.296183 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:07.295731    2563 scope.go:117] "RemoveContainer" containerID="9a70f615b878e4be5935f9fa88713b007e7b9ee1f8948001dce968a2450b9c85"
Sep 14 08:04:07.297102 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:07.297086    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"
Sep 14 08:04:07.297102 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:07.297094    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9" event={"ID":"6c78d8a3-2056-488e-a72e-f05a727c54d7","Type":"ContainerDied","Data":"58a85b00d8798069f5859b180d013151fc93f7a619af7492b4d16bd207cdd6be"}
Sep 14 08:04:07.306539 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:07.306257    2563 scope.go:117] "RemoveContainer" containerID="1c1d7f27465aa3e190d415ed8c4988f21b15000a7416b9d07db411cff97fdb8d"
Sep 14 08:04:07.328849 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:07.328824    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"]
Sep 14 08:04:07.335424 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:07.335401    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-814672-9bb45769d-k4b9h"]
Sep 14 08:04:07.347650 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:07.347613    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"]
Sep 14 08:04:07.354163 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:07.354138    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-814672-74d9548755-hpck9"]
Sep 14 08:04:07.886498 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:07.886462    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d0ade40-04c9-4c1b-be18-ef543d09911b" path="/var/lib/kubelet/pods/0d0ade40-04c9-4c1b-be18-ef543d09911b/volumes"
Sep 14 08:04:07.886853 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:07.886837    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c78d8a3-2056-488e-a72e-f05a727c54d7" path="/var/lib/kubelet/pods/6c78d8a3-2056-488e-a72e-f05a727c54d7/volumes"
Sep 14 08:04:07.887155 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:07.887143    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78e9b78a-6eeb-4c01-a01c-beb29edf387b" path="/var/lib/kubelet/pods/78e9b78a-6eeb-4c01-a01c-beb29edf387b/volumes"
Sep 14 08:04:33.721253 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.721169    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"]
Sep 14 08:04:33.721721 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.721600    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c78d8a3-2056-488e-a72e-f05a727c54d7" containerName="payload-processing"
Sep 14 08:04:33.721721 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.721612    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c78d8a3-2056-488e-a72e-f05a727c54d7" containerName="payload-processing"
Sep 14 08:04:33.721721 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.721630    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d0ade40-04c9-4c1b-be18-ef543d09911b" containerName="revoke-keys"
Sep 14 08:04:33.721721 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.721635    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d0ade40-04c9-4c1b-be18-ef543d09911b" containerName="revoke-keys"
Sep 14 08:04:33.721721 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.721676    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e9b78a-6eeb-4c01-a01c-beb29edf387b" containerName="payload-pre-processing"
Sep 14 08:04:33.721721 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.721682    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e9b78a-6eeb-4c01-a01c-beb29edf387b" containerName="payload-pre-processing"
Sep 14 08:04:33.721947 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.721752    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c78d8a3-2056-488e-a72e-f05a727c54d7" containerName="payload-processing"
Sep 14 08:04:33.721947 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.721760    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d0ade40-04c9-4c1b-be18-ef543d09911b" containerName="revoke-keys"
Sep 14 08:04:33.721947 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.721769    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="78e9b78a-6eeb-4c01-a01c-beb29edf387b" containerName="payload-pre-processing"
Sep 14 08:04:33.725104 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.725088    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"
Sep 14 08:04:33.728474 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.728444    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-73bd4a\""
Sep 14 08:04:33.755204 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.755175    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"]
Sep 14 08:04:33.792990 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.792959    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"]
Sep 14 08:04:33.796881 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.796863    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"
Sep 14 08:04:33.799478 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.799462    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-73bd4a-dockercfg-fmhbz\""
Sep 14 08:04:33.812358 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.812335    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"]
Sep 14 08:04:33.825701 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.825630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/739afdbf-b567-4d2f-9553-82749d1a7cf7-plugins-config-volume\") pod \"payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw\" (UID: \"739afdbf-b567-4d2f-9553-82749d1a7cf7\") " pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"
Sep 14 08:04:33.825790 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.825766    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmblc\" (UniqueName: \"kubernetes.io/projected/739afdbf-b567-4d2f-9553-82749d1a7cf7-kube-api-access-lmblc\") pod \"payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw\" (UID: \"739afdbf-b567-4d2f-9553-82749d1a7cf7\") " pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"
Sep 14 08:04:33.927081 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.927045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wntqj\" (UniqueName: \"kubernetes.io/projected/7d998131-622d-4d96-808e-e6341bf0f52c-kube-api-access-wntqj\") pod \"payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr\" (UID: \"7d998131-622d-4d96-808e-e6341bf0f52c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"
Sep 14 08:04:33.927081 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.927082    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d998131-622d-4d96-808e-e6341bf0f52c-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr\" (UID: \"7d998131-622d-4d96-808e-e6341bf0f52c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"
Sep 14 08:04:33.927328 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.927158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/739afdbf-b567-4d2f-9553-82749d1a7cf7-plugins-config-volume\") pod \"payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw\" (UID: \"739afdbf-b567-4d2f-9553-82749d1a7cf7\") " pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"
Sep 14 08:04:33.927328 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.927260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmblc\" (UniqueName: \"kubernetes.io/projected/739afdbf-b567-4d2f-9553-82749d1a7cf7-kube-api-access-lmblc\") pod \"payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw\" (UID: \"739afdbf-b567-4d2f-9553-82749d1a7cf7\") " pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"
Sep 14 08:04:33.927864 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.927845    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/739afdbf-b567-4d2f-9553-82749d1a7cf7-plugins-config-volume\") pod \"payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw\" (UID: \"739afdbf-b567-4d2f-9553-82749d1a7cf7\") " pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"
Sep 14 08:04:33.935633 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:33.935601    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmblc\" (UniqueName: \"kubernetes.io/projected/739afdbf-b567-4d2f-9553-82749d1a7cf7-kube-api-access-lmblc\") pod \"payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw\" (UID: \"739afdbf-b567-4d2f-9553-82749d1a7cf7\") " pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"
Sep 14 08:04:34.027977 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.027914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wntqj\" (UniqueName: \"kubernetes.io/projected/7d998131-622d-4d96-808e-e6341bf0f52c-kube-api-access-wntqj\") pod \"payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr\" (UID: \"7d998131-622d-4d96-808e-e6341bf0f52c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"
Sep 14 08:04:34.027977 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.027944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d998131-622d-4d96-808e-e6341bf0f52c-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr\" (UID: \"7d998131-622d-4d96-808e-e6341bf0f52c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"
Sep 14 08:04:34.028674 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.028633    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d998131-622d-4d96-808e-e6341bf0f52c-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr\" (UID: \"7d998131-622d-4d96-808e-e6341bf0f52c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"
Sep 14 08:04:34.035807 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.035788    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"
Sep 14 08:04:34.049549 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.049528    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wntqj\" (UniqueName: \"kubernetes.io/projected/7d998131-622d-4d96-808e-e6341bf0f52c-kube-api-access-wntqj\") pod \"payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr\" (UID: \"7d998131-622d-4d96-808e-e6341bf0f52c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"
Sep 14 08:04:34.107170 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.106955    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"
Sep 14 08:04:34.315428 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.315399    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"]
Sep 14 08:04:34.316773 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:04:34.316744    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod739afdbf_b567_4d2f_9553_82749d1a7cf7.slice/crio-a80e8fb14f816ac834e99b1edf8500c95eb39378dbf68a746c33569268c80587 WatchSource:0}: Error finding container a80e8fb14f816ac834e99b1edf8500c95eb39378dbf68a746c33569268c80587: Status 404 returned error can't find the container with id a80e8fb14f816ac834e99b1edf8500c95eb39378dbf68a746c33569268c80587
Sep 14 08:04:34.388219 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.388192    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"]
Sep 14 08:04:34.389809 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:04:34.389786    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d998131_622d_4d96_808e_e6341bf0f52c.slice/crio-b87f488c092d36511061f969814bb31d3989d5b1cb0f6dd938b3fc6e7a0fd221 WatchSource:0}: Error finding container b87f488c092d36511061f969814bb31d3989d5b1cb0f6dd938b3fc6e7a0fd221: Status 404 returned error can't find the container with id b87f488c092d36511061f969814bb31d3989d5b1cb0f6dd938b3fc6e7a0fd221
Sep 14 08:04:34.469509 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.469474    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr" event={"ID":"7d998131-622d-4d96-808e-e6341bf0f52c","Type":"ContainerStarted","Data":"9b33c984930ded6ffadd335438eecf6d8fde35dec40859ac58989735758e4c81"}
Sep 14 08:04:34.469678 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.469541    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr" event={"ID":"7d998131-622d-4d96-808e-e6341bf0f52c","Type":"ContainerStarted","Data":"b87f488c092d36511061f969814bb31d3989d5b1cb0f6dd938b3fc6e7a0fd221"}
Sep 14 08:04:34.469678 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.469653    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"
Sep 14 08:04:34.471292 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.471263    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw" event={"ID":"739afdbf-b567-4d2f-9553-82749d1a7cf7","Type":"ContainerStarted","Data":"bc2a0b4f14e919266205a4c890fb42974dc954ded9a21a81472f48b7f33398dc"}
Sep 14 08:04:34.471593 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.471318    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw" event={"ID":"739afdbf-b567-4d2f-9553-82749d1a7cf7","Type":"ContainerStarted","Data":"a80e8fb14f816ac834e99b1edf8500c95eb39378dbf68a746c33569268c80587"}
Sep 14 08:04:34.471593 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.471345    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"
Sep 14 08:04:34.489354 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.489311    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr" podStartSLOduration=1.489297653 podStartE2EDuration="1.489297653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:34.486782663 +0000 UTC m=+995.115670314" watchObservedRunningTime="2026-09-14 08:04:34.489297653 +0000 UTC m=+995.118185311"
Sep 14 08:04:34.509224 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:34.509175    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw" podStartSLOduration=1.509155845 podStartE2EDuration="1.509155845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:34.504335679 +0000 UTC m=+995.133223307" watchObservedRunningTime="2026-09-14 08:04:34.509155845 +0000 UTC m=+995.138043476"
Sep 14 08:04:45.479683 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:45.479654    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"
Sep 14 08:04:45.480125 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:04:45.479860    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"
Sep 14 08:05:08.779288 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:08.779247    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"]
Sep 14 08:05:08.786013 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:08.785989    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"
Sep 14 08:05:08.789549 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:08.789525    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-af014a32-dockercfg-m9c7f\""
Sep 14 08:05:08.789707 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:08.789671    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-af014a32\""
Sep 14 08:05:08.810099 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:08.810072    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"]
Sep 14 08:05:08.846316 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:08.846288    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"]
Sep 14 08:05:08.849866 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:08.849848    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"
Sep 14 08:05:08.863249 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:08.863230    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"]
Sep 14 08:05:08.953061 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:08.953030    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0c48311-2c4a-47ea-b04d-f377c84b2404-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62\" (UID: \"d0c48311-2c4a-47ea-b04d-f377c84b2404\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"
Sep 14 08:05:08.953214 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:08.953066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b934e50-5383-4581-9cd9-97105ccc318e-plugins-config-volume\") pod \"payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v\" (UID: \"4b934e50-5383-4581-9cd9-97105ccc318e\") " pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"
Sep 14 08:05:08.953214 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:08.953099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q5gq\" (UniqueName: \"kubernetes.io/projected/4b934e50-5383-4581-9cd9-97105ccc318e-kube-api-access-5q5gq\") pod \"payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v\" (UID: \"4b934e50-5383-4581-9cd9-97105ccc318e\") " pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"
Sep 14 08:05:08.953214 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:08.953146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk7dw\" (UniqueName: \"kubernetes.io/projected/d0c48311-2c4a-47ea-b04d-f377c84b2404-kube-api-access-kk7dw\") pod \"payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62\" (UID: \"d0c48311-2c4a-47ea-b04d-f377c84b2404\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"
Sep 14 08:05:09.053773 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.053691    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0c48311-2c4a-47ea-b04d-f377c84b2404-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62\" (UID: \"d0c48311-2c4a-47ea-b04d-f377c84b2404\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"
Sep 14 08:05:09.053773 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.053731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b934e50-5383-4581-9cd9-97105ccc318e-plugins-config-volume\") pod \"payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v\" (UID: \"4b934e50-5383-4581-9cd9-97105ccc318e\") " pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"
Sep 14 08:05:09.053773 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.053767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q5gq\" (UniqueName: \"kubernetes.io/projected/4b934e50-5383-4581-9cd9-97105ccc318e-kube-api-access-5q5gq\") pod \"payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v\" (UID: \"4b934e50-5383-4581-9cd9-97105ccc318e\") " pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"
Sep 14 08:05:09.054046 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.053782    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk7dw\" (UniqueName: \"kubernetes.io/projected/d0c48311-2c4a-47ea-b04d-f377c84b2404-kube-api-access-kk7dw\") pod \"payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62\" (UID: \"d0c48311-2c4a-47ea-b04d-f377c84b2404\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"
Sep 14 08:05:09.054481 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.054459    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0c48311-2c4a-47ea-b04d-f377c84b2404-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62\" (UID: \"d0c48311-2c4a-47ea-b04d-f377c84b2404\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"
Sep 14 08:05:09.054574 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.054517    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b934e50-5383-4581-9cd9-97105ccc318e-plugins-config-volume\") pod \"payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v\" (UID: \"4b934e50-5383-4581-9cd9-97105ccc318e\") " pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"
Sep 14 08:05:09.061716 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.061693    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk7dw\" (UniqueName: \"kubernetes.io/projected/d0c48311-2c4a-47ea-b04d-f377c84b2404-kube-api-access-kk7dw\") pod \"payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62\" (UID: \"d0c48311-2c4a-47ea-b04d-f377c84b2404\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"
Sep 14 08:05:09.061812 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.061781    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q5gq\" (UniqueName: \"kubernetes.io/projected/4b934e50-5383-4581-9cd9-97105ccc318e-kube-api-access-5q5gq\") pod \"payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v\" (UID: \"4b934e50-5383-4581-9cd9-97105ccc318e\") " pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"
Sep 14 08:05:09.097331 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.097306    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"
Sep 14 08:05:09.159245 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.159217    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"
Sep 14 08:05:09.371201 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.371169    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"]
Sep 14 08:05:09.372968 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:05:09.372941    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b934e50_5383_4581_9cd9_97105ccc318e.slice/crio-33ac21f87e7df85c0a6167a062970a045e12df3d4311ed325db077a71617f94e WatchSource:0}: Error finding container 33ac21f87e7df85c0a6167a062970a045e12df3d4311ed325db077a71617f94e: Status 404 returned error can't find the container with id 33ac21f87e7df85c0a6167a062970a045e12df3d4311ed325db077a71617f94e
Sep 14 08:05:09.442118 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:05:09.442088    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c48311_2c4a_47ea_b04d_f377c84b2404.slice/crio-8b164e1ee07d5ff5f6f426fc95f29c86eea8de5f33f00d02a6404aa308a702bf WatchSource:0}: Error finding container 8b164e1ee07d5ff5f6f426fc95f29c86eea8de5f33f00d02a6404aa308a702bf: Status 404 returned error can't find the container with id 8b164e1ee07d5ff5f6f426fc95f29c86eea8de5f33f00d02a6404aa308a702bf
Sep 14 08:05:09.442741 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.442690    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"]
Sep 14 08:05:09.692966 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.692878    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v" event={"ID":"4b934e50-5383-4581-9cd9-97105ccc318e","Type":"ContainerStarted","Data":"4724cf31407e7ae9aaf51beb5de15fb9be8f9ff5867d85bf5f6698ffbb4b135c"}
Sep 14 08:05:09.692966 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.692939    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v" event={"ID":"4b934e50-5383-4581-9cd9-97105ccc318e","Type":"ContainerStarted","Data":"33ac21f87e7df85c0a6167a062970a045e12df3d4311ed325db077a71617f94e"}
Sep 14 08:05:09.693190 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.693125    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"
Sep 14 08:05:09.694688 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.694667    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62" event={"ID":"d0c48311-2c4a-47ea-b04d-f377c84b2404","Type":"ContainerStarted","Data":"1cc2d6b72b7672b70d964133f59ea6077b22b9419d1dd9b7edb1465b1214bcde"}
Sep 14 08:05:09.694805 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.694708    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62" event={"ID":"d0c48311-2c4a-47ea-b04d-f377c84b2404","Type":"ContainerStarted","Data":"8b164e1ee07d5ff5f6f426fc95f29c86eea8de5f33f00d02a6404aa308a702bf"}
Sep 14 08:05:09.694805 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.694759    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"
Sep 14 08:05:09.710651 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.710603    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v" podStartSLOduration=1.710589325 podStartE2EDuration="1.710589325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:05:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:05:09.708252488 +0000 UTC m=+1030.337140135" watchObservedRunningTime="2026-09-14 08:05:09.710589325 +0000 UTC m=+1030.339476953"
Sep 14 08:05:09.725747 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:09.725712    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62" podStartSLOduration=1.72569916 podStartE2EDuration="1.72569916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:05:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:05:09.723290645 +0000 UTC m=+1030.352178307" watchObservedRunningTime="2026-09-14 08:05:09.72569916 +0000 UTC m=+1030.354586787"
Sep 14 08:05:20.702518 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:20.702487    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"
Sep 14 08:05:20.702902 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:20.702553    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"
Sep 14 08:05:52.748972 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:52.748882    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"]
Sep 14 08:05:52.749766 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:52.749123    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v" podUID="4b934e50-5383-4581-9cd9-97105ccc318e" containerName="payload-processing" containerID="cri-o://4724cf31407e7ae9aaf51beb5de15fb9be8f9ff5867d85bf5f6698ffbb4b135c" gracePeriod=30
Sep 14 08:05:52.755125 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:52.755097    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"]
Sep 14 08:05:52.755344 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:52.755303    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62" podUID="d0c48311-2c4a-47ea-b04d-f377c84b2404" containerName="payload-pre-processing" containerID="cri-o://1cc2d6b72b7672b70d964133f59ea6077b22b9419d1dd9b7edb1465b1214bcde" gracePeriod=30
Sep 14 08:05:52.964304 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:52.964043    2563 generic.go:356] "Generic (PLEG): container finished" podID="4b934e50-5383-4581-9cd9-97105ccc318e" containerID="4724cf31407e7ae9aaf51beb5de15fb9be8f9ff5867d85bf5f6698ffbb4b135c" exitCode=0
Sep 14 08:05:52.964304 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:52.964239    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v" event={"ID":"4b934e50-5383-4581-9cd9-97105ccc318e","Type":"ContainerDied","Data":"4724cf31407e7ae9aaf51beb5de15fb9be8f9ff5867d85bf5f6698ffbb4b135c"}
Sep 14 08:05:52.966911 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:52.966620    2563 generic.go:356] "Generic (PLEG): container finished" podID="d0c48311-2c4a-47ea-b04d-f377c84b2404" containerID="1cc2d6b72b7672b70d964133f59ea6077b22b9419d1dd9b7edb1465b1214bcde" exitCode=0
Sep 14 08:05:52.966911 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:52.966776    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62" event={"ID":"d0c48311-2c4a-47ea-b04d-f377c84b2404","Type":"ContainerDied","Data":"1cc2d6b72b7672b70d964133f59ea6077b22b9419d1dd9b7edb1465b1214bcde"}
Sep 14 08:05:53.019931 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.019907    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"
Sep 14 08:05:53.052527 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.052500    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"
Sep 14 08:05:53.147392 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.147357    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5q5gq\" (UniqueName: \"kubernetes.io/projected/4b934e50-5383-4581-9cd9-97105ccc318e-kube-api-access-5q5gq\") pod \"4b934e50-5383-4581-9cd9-97105ccc318e\" (UID: \"4b934e50-5383-4581-9cd9-97105ccc318e\") "
Sep 14 08:05:53.147583 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.147410    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0c48311-2c4a-47ea-b04d-f377c84b2404-plugins-config-volume\") pod \"d0c48311-2c4a-47ea-b04d-f377c84b2404\" (UID: \"d0c48311-2c4a-47ea-b04d-f377c84b2404\") "
Sep 14 08:05:53.147583 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.147470    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kk7dw\" (UniqueName: \"kubernetes.io/projected/d0c48311-2c4a-47ea-b04d-f377c84b2404-kube-api-access-kk7dw\") pod \"d0c48311-2c4a-47ea-b04d-f377c84b2404\" (UID: \"d0c48311-2c4a-47ea-b04d-f377c84b2404\") "
Sep 14 08:05:53.147583 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.147503    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b934e50-5383-4581-9cd9-97105ccc318e-plugins-config-volume\") pod \"4b934e50-5383-4581-9cd9-97105ccc318e\" (UID: \"4b934e50-5383-4581-9cd9-97105ccc318e\") "
Sep 14 08:05:53.147949 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.147912    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0c48311-2c4a-47ea-b04d-f377c84b2404-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0c48311-2c4a-47ea-b04d-f377c84b2404" (UID: "d0c48311-2c4a-47ea-b04d-f377c84b2404"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:53.148074 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.147974    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b934e50-5383-4581-9cd9-97105ccc318e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4b934e50-5383-4581-9cd9-97105ccc318e" (UID: "4b934e50-5383-4581-9cd9-97105ccc318e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:53.149657 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.149620    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b934e50-5383-4581-9cd9-97105ccc318e-kube-api-access-5q5gq" (OuterVolumeSpecName: "kube-api-access-5q5gq") pod "4b934e50-5383-4581-9cd9-97105ccc318e" (UID: "4b934e50-5383-4581-9cd9-97105ccc318e"). InnerVolumeSpecName "kube-api-access-5q5gq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:05:53.149753 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.149665    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0c48311-2c4a-47ea-b04d-f377c84b2404-kube-api-access-kk7dw" (OuterVolumeSpecName: "kube-api-access-kk7dw") pod "d0c48311-2c4a-47ea-b04d-f377c84b2404" (UID: "d0c48311-2c4a-47ea-b04d-f377c84b2404"). InnerVolumeSpecName "kube-api-access-kk7dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:05:53.248705 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.248675    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0c48311-2c4a-47ea-b04d-f377c84b2404-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:05:53.248705 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.248703    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk7dw\" (UniqueName: \"kubernetes.io/projected/d0c48311-2c4a-47ea-b04d-f377c84b2404-kube-api-access-kk7dw\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:05:53.248705 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.248714    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b934e50-5383-4581-9cd9-97105ccc318e-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:05:53.248932 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.248723    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q5gq\" (UniqueName: \"kubernetes.io/projected/4b934e50-5383-4581-9cd9-97105ccc318e-kube-api-access-5q5gq\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:05:53.973854 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.973826    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"
Sep 14 08:05:53.974208 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.973864    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62" event={"ID":"d0c48311-2c4a-47ea-b04d-f377c84b2404","Type":"ContainerDied","Data":"8b164e1ee07d5ff5f6f426fc95f29c86eea8de5f33f00d02a6404aa308a702bf"}
Sep 14 08:05:53.974208 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.973924    2563 scope.go:117] "RemoveContainer" containerID="1cc2d6b72b7672b70d964133f59ea6077b22b9419d1dd9b7edb1465b1214bcde"
Sep 14 08:05:53.975302 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.975263    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v" event={"ID":"4b934e50-5383-4581-9cd9-97105ccc318e","Type":"ContainerDied","Data":"33ac21f87e7df85c0a6167a062970a045e12df3d4311ed325db077a71617f94e"}
Sep 14 08:05:53.975302 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.975280    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"
Sep 14 08:05:53.986125 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.986063    2563 scope.go:117] "RemoveContainer" containerID="4724cf31407e7ae9aaf51beb5de15fb9be8f9ff5867d85bf5f6698ffbb4b135c"
Sep 14 08:05:54.000656 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:53.999976    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"]
Sep 14 08:05:54.006770 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:54.006749    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-af014a32-7cbc95fcfd-xkt62"]
Sep 14 08:05:54.017656 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:54.017592    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"]
Sep 14 08:05:54.024107 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:54.024086    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-af014a32-7479bf4c77-s9s5v"]
Sep 14 08:05:55.886663 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:55.886611    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b934e50-5383-4581-9cd9-97105ccc318e" path="/var/lib/kubelet/pods/4b934e50-5383-4581-9cd9-97105ccc318e/volumes"
Sep 14 08:05:55.887028 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:05:55.886991    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0c48311-2c4a-47ea-b04d-f377c84b2404" path="/var/lib/kubelet/pods/d0c48311-2c4a-47ea-b04d-f377c84b2404/volumes"
Sep 14 08:06:20.317052 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.317015    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"]
Sep 14 08:06:20.317720 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.317699    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0c48311-2c4a-47ea-b04d-f377c84b2404" containerName="payload-pre-processing"
Sep 14 08:06:20.317720 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.317721    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0c48311-2c4a-47ea-b04d-f377c84b2404" containerName="payload-pre-processing"
Sep 14 08:06:20.317901 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.317744    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b934e50-5383-4581-9cd9-97105ccc318e" containerName="payload-processing"
Sep 14 08:06:20.317901 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.317752    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b934e50-5383-4581-9cd9-97105ccc318e" containerName="payload-processing"
Sep 14 08:06:20.317901 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.317887    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b934e50-5383-4581-9cd9-97105ccc318e" containerName="payload-processing"
Sep 14 08:06:20.318058 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.317904    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0c48311-2c4a-47ea-b04d-f377c84b2404" containerName="payload-pre-processing"
Sep 14 08:06:20.322273 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.322254    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"
Sep 14 08:06:20.324451 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.324429    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-b990f4\""
Sep 14 08:06:20.324571 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.324499    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-b990f4-dockercfg-q6swf\""
Sep 14 08:06:20.330608 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.330590    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"]
Sep 14 08:06:20.340978 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.340953    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"]
Sep 14 08:06:20.348000 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.347969    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"
Sep 14 08:06:20.352530 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.352505    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"]
Sep 14 08:06:20.410967 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.410937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b54011f6-c356-408a-bb06-41d0e43121e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt\" (UID: \"b54011f6-c356-408a-bb06-41d0e43121e9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"
Sep 14 08:06:20.411110 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.410990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e20d590-36f5-443f-838f-6696ccefd7a3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr\" (UID: \"8e20d590-36f5-443f-838f-6696ccefd7a3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"
Sep 14 08:06:20.411110 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.411010    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6rhq\" (UniqueName: \"kubernetes.io/projected/b54011f6-c356-408a-bb06-41d0e43121e9-kube-api-access-g6rhq\") pod \"payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt\" (UID: \"b54011f6-c356-408a-bb06-41d0e43121e9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"
Sep 14 08:06:20.411110 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.411064    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx8mt\" (UniqueName: \"kubernetes.io/projected/8e20d590-36f5-443f-838f-6696ccefd7a3-kube-api-access-zx8mt\") pod \"payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr\" (UID: \"8e20d590-36f5-443f-838f-6696ccefd7a3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"
Sep 14 08:06:20.511787 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.511754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b54011f6-c356-408a-bb06-41d0e43121e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt\" (UID: \"b54011f6-c356-408a-bb06-41d0e43121e9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"
Sep 14 08:06:20.511973 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.511809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e20d590-36f5-443f-838f-6696ccefd7a3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr\" (UID: \"8e20d590-36f5-443f-838f-6696ccefd7a3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"
Sep 14 08:06:20.511973 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.511831    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6rhq\" (UniqueName: \"kubernetes.io/projected/b54011f6-c356-408a-bb06-41d0e43121e9-kube-api-access-g6rhq\") pod \"payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt\" (UID: \"b54011f6-c356-408a-bb06-41d0e43121e9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"
Sep 14 08:06:20.511973 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.511853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx8mt\" (UniqueName: \"kubernetes.io/projected/8e20d590-36f5-443f-838f-6696ccefd7a3-kube-api-access-zx8mt\") pod \"payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr\" (UID: \"8e20d590-36f5-443f-838f-6696ccefd7a3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"
Sep 14 08:06:20.512683 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.512657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e20d590-36f5-443f-838f-6696ccefd7a3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr\" (UID: \"8e20d590-36f5-443f-838f-6696ccefd7a3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"
Sep 14 08:06:20.512762 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.512679    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b54011f6-c356-408a-bb06-41d0e43121e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt\" (UID: \"b54011f6-c356-408a-bb06-41d0e43121e9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"
Sep 14 08:06:20.519632 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.519604    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx8mt\" (UniqueName: \"kubernetes.io/projected/8e20d590-36f5-443f-838f-6696ccefd7a3-kube-api-access-zx8mt\") pod \"payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr\" (UID: \"8e20d590-36f5-443f-838f-6696ccefd7a3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"
Sep 14 08:06:20.519769 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.519748    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6rhq\" (UniqueName: \"kubernetes.io/projected/b54011f6-c356-408a-bb06-41d0e43121e9-kube-api-access-g6rhq\") pod \"payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt\" (UID: \"b54011f6-c356-408a-bb06-41d0e43121e9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"
Sep 14 08:06:20.634718 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.634681    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"
Sep 14 08:06:20.659686 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.659660    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"
Sep 14 08:06:20.912921 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.912893    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"]
Sep 14 08:06:20.914003 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:06:20.913972    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb54011f6_c356_408a_bb06_41d0e43121e9.slice/crio-66ae9a0568b36f59c6886c0b208275b9914ed6e408daac9d13674c9078ef31d9 WatchSource:0}: Error finding container 66ae9a0568b36f59c6886c0b208275b9914ed6e408daac9d13674c9078ef31d9: Status 404 returned error can't find the container with id 66ae9a0568b36f59c6886c0b208275b9914ed6e408daac9d13674c9078ef31d9
Sep 14 08:06:20.942225 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:20.942200    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"]
Sep 14 08:06:20.946878 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:06:20.946845    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e20d590_36f5_443f_838f_6696ccefd7a3.slice/crio-5a8234b4c1284aabda15846735d83bb8b1be64b927e830addc2f4fc5e9410532 WatchSource:0}: Error finding container 5a8234b4c1284aabda15846735d83bb8b1be64b927e830addc2f4fc5e9410532: Status 404 returned error can't find the container with id 5a8234b4c1284aabda15846735d83bb8b1be64b927e830addc2f4fc5e9410532
Sep 14 08:06:21.138445 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:21.138404    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr" event={"ID":"8e20d590-36f5-443f-838f-6696ccefd7a3","Type":"ContainerStarted","Data":"e7d13bb37bc4ffdb7a3179be47c78f81f1dafff9101ecde90d0cb8531a07fd72"}
Sep 14 08:06:21.138684 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:21.138477    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"
Sep 14 08:06:21.138684 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:21.138503    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr" event={"ID":"8e20d590-36f5-443f-838f-6696ccefd7a3","Type":"ContainerStarted","Data":"5a8234b4c1284aabda15846735d83bb8b1be64b927e830addc2f4fc5e9410532"}
Sep 14 08:06:21.140334 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:21.140300    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt" event={"ID":"b54011f6-c356-408a-bb06-41d0e43121e9","Type":"ContainerStarted","Data":"59115d7f3c5628f941bf25ca92f8c802fca67fe4d95a4657b8e314f359e3764a"}
Sep 14 08:06:21.140453 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:21.140355    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt" event={"ID":"b54011f6-c356-408a-bb06-41d0e43121e9","Type":"ContainerStarted","Data":"66ae9a0568b36f59c6886c0b208275b9914ed6e408daac9d13674c9078ef31d9"}
Sep 14 08:06:21.140550 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:21.140524    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"
Sep 14 08:06:21.157032 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:21.156982    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr" podStartSLOduration=1.156963891 podStartE2EDuration="1.156963891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:21.155854344 +0000 UTC m=+1101.784741990" watchObservedRunningTime="2026-09-14 08:06:21.156963891 +0000 UTC m=+1101.785851520"
Sep 14 08:06:21.173296 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:21.173211    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt" podStartSLOduration=1.173192592 podStartE2EDuration="1.173192592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:21.169365904 +0000 UTC m=+1101.798253556" watchObservedRunningTime="2026-09-14 08:06:21.173192592 +0000 UTC m=+1101.802080223"
Sep 14 08:06:32.152372 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:32.152340    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"
Sep 14 08:06:32.152862 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:32.152412    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"
Sep 14 08:06:55.370573 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.370538    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"]
Sep 14 08:06:55.375364 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.375348    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"
Sep 14 08:06:55.377792 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.377767    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-afcf8a\""
Sep 14 08:06:55.385372 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.385343    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"]
Sep 14 08:06:55.392846 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.392824    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"]
Sep 14 08:06:55.396589 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.396569    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"
Sep 14 08:06:55.398946 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.398927    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-afcf8a-dockercfg-tcjkx\""
Sep 14 08:06:55.407161 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.407139    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"]
Sep 14 08:06:55.423906 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.423874    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvz4d\" (UniqueName: \"kubernetes.io/projected/86f0e8a6-58bd-46d0-9003-1d9d113d22a8-kube-api-access-xvz4d\") pod \"payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw\" (UID: \"86f0e8a6-58bd-46d0-9003-1d9d113d22a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"
Sep 14 08:06:55.424029 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.423911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09bd8e46-b966-401a-abac-7ef008d8c051-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn\" (UID: \"09bd8e46-b966-401a-abac-7ef008d8c051\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"
Sep 14 08:06:55.424029 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.423942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdv4p\" (UniqueName: \"kubernetes.io/projected/09bd8e46-b966-401a-abac-7ef008d8c051-kube-api-access-mdv4p\") pod \"payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn\" (UID: \"09bd8e46-b966-401a-abac-7ef008d8c051\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"
Sep 14 08:06:55.424029 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.424020    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86f0e8a6-58bd-46d0-9003-1d9d113d22a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw\" (UID: \"86f0e8a6-58bd-46d0-9003-1d9d113d22a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"
Sep 14 08:06:55.525158 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.525104    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdv4p\" (UniqueName: \"kubernetes.io/projected/09bd8e46-b966-401a-abac-7ef008d8c051-kube-api-access-mdv4p\") pod \"payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn\" (UID: \"09bd8e46-b966-401a-abac-7ef008d8c051\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"
Sep 14 08:06:55.525310 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.525203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86f0e8a6-58bd-46d0-9003-1d9d113d22a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw\" (UID: \"86f0e8a6-58bd-46d0-9003-1d9d113d22a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"
Sep 14 08:06:55.525357 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.525329    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvz4d\" (UniqueName: \"kubernetes.io/projected/86f0e8a6-58bd-46d0-9003-1d9d113d22a8-kube-api-access-xvz4d\") pod \"payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw\" (UID: \"86f0e8a6-58bd-46d0-9003-1d9d113d22a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"
Sep 14 08:06:55.525402 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.525353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09bd8e46-b966-401a-abac-7ef008d8c051-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn\" (UID: \"09bd8e46-b966-401a-abac-7ef008d8c051\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"
Sep 14 08:06:55.526080 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.526054    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86f0e8a6-58bd-46d0-9003-1d9d113d22a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw\" (UID: \"86f0e8a6-58bd-46d0-9003-1d9d113d22a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"
Sep 14 08:06:55.526203 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.526059    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09bd8e46-b966-401a-abac-7ef008d8c051-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn\" (UID: \"09bd8e46-b966-401a-abac-7ef008d8c051\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"
Sep 14 08:06:55.534104 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.534070    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdv4p\" (UniqueName: \"kubernetes.io/projected/09bd8e46-b966-401a-abac-7ef008d8c051-kube-api-access-mdv4p\") pod \"payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn\" (UID: \"09bd8e46-b966-401a-abac-7ef008d8c051\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"
Sep 14 08:06:55.534750 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.534723    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvz4d\" (UniqueName: \"kubernetes.io/projected/86f0e8a6-58bd-46d0-9003-1d9d113d22a8-kube-api-access-xvz4d\") pod \"payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw\" (UID: \"86f0e8a6-58bd-46d0-9003-1d9d113d22a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"
Sep 14 08:06:55.688576 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.688488    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"
Sep 14 08:06:55.707577 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.707554    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"
Sep 14 08:06:55.968331 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.968303    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"]
Sep 14 08:06:55.968728 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:06:55.968705    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86f0e8a6_58bd_46d0_9003_1d9d113d22a8.slice/crio-f7f2d2ba44fdf5c733ca4ea6a80d6a3372ef4125af135b9b21366ac84b0cec63 WatchSource:0}: Error finding container f7f2d2ba44fdf5c733ca4ea6a80d6a3372ef4125af135b9b21366ac84b0cec63: Status 404 returned error can't find the container with id f7f2d2ba44fdf5c733ca4ea6a80d6a3372ef4125af135b9b21366ac84b0cec63
Sep 14 08:06:55.998998 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:55.998975    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"]
Sep 14 08:06:56.005754 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:06:56.005732    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09bd8e46_b966_401a_abac_7ef008d8c051.slice/crio-0ae2eae4ad320aa160e3f6afe2fdf9db32b0ec6a4415e7a3906b96900a4b9eb6 WatchSource:0}: Error finding container 0ae2eae4ad320aa160e3f6afe2fdf9db32b0ec6a4415e7a3906b96900a4b9eb6: Status 404 returned error can't find the container with id 0ae2eae4ad320aa160e3f6afe2fdf9db32b0ec6a4415e7a3906b96900a4b9eb6
Sep 14 08:06:56.360744 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:56.360702    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw" event={"ID":"86f0e8a6-58bd-46d0-9003-1d9d113d22a8","Type":"ContainerStarted","Data":"72c6abfafae5df1b572c3cd58101157315ae20bf96a975bc55c2f4d15bfd914a"}
Sep 14 08:06:56.360919 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:56.360773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw" event={"ID":"86f0e8a6-58bd-46d0-9003-1d9d113d22a8","Type":"ContainerStarted","Data":"f7f2d2ba44fdf5c733ca4ea6a80d6a3372ef4125af135b9b21366ac84b0cec63"}
Sep 14 08:06:56.361027 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:56.360916    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"
Sep 14 08:06:56.363059 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:56.363033    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn" event={"ID":"09bd8e46-b966-401a-abac-7ef008d8c051","Type":"ContainerStarted","Data":"d0acdb082eae2a14d3da10da7e8bc29f8187417351a9397411e3c7a57f896907"}
Sep 14 08:06:56.363195 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:56.363078    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn" event={"ID":"09bd8e46-b966-401a-abac-7ef008d8c051","Type":"ContainerStarted","Data":"0ae2eae4ad320aa160e3f6afe2fdf9db32b0ec6a4415e7a3906b96900a4b9eb6"}
Sep 14 08:06:56.363195 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:56.363113    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"
Sep 14 08:06:56.377504 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:56.377463    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw" podStartSLOduration=1.377450642 podStartE2EDuration="1.377450642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:56.375263238 +0000 UTC m=+1137.004150880" watchObservedRunningTime="2026-09-14 08:06:56.377450642 +0000 UTC m=+1137.006338268"
Sep 14 08:06:56.393494 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:06:56.393454    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn" podStartSLOduration=1.39344245 podStartE2EDuration="1.39344245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:56.390789686 +0000 UTC m=+1137.019677334" watchObservedRunningTime="2026-09-14 08:06:56.39344245 +0000 UTC m=+1137.022330110"
Sep 14 08:07:07.018469 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:07.018433    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq"]
Sep 14 08:07:07.022433 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:07.022413    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq"
Sep 14 08:07:07.024508 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:07.024491    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:07:07.029434 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:07.029408    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq"]
Sep 14 08:07:07.132147 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:07.132099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf169ea-91c0-4869-be34-d9a3e39128d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq\" (UID: \"acf169ea-91c0-4869-be34-d9a3e39128d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq"
Sep 14 08:07:07.232882 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:07.232847    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf169ea-91c0-4869-be34-d9a3e39128d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq\" (UID: \"acf169ea-91c0-4869-be34-d9a3e39128d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq"
Sep 14 08:07:07.233470 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:07.233450    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf169ea-91c0-4869-be34-d9a3e39128d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq\" (UID: \"acf169ea-91c0-4869-be34-d9a3e39128d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq"
Sep 14 08:07:07.334262 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:07.334236    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq"
Sep 14 08:07:07.371681 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:07.371632    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"
Sep 14 08:07:07.371894 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:07.371873    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"
Sep 14 08:07:07.612887 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:07.612860    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq"]
Sep 14 08:07:07.613876 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:07:07.613845    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf169ea_91c0_4869_be34_d9a3e39128d5.slice/crio-e5d2c0f1bb1b02e4b28459e24b78633894076beb8b8bed393cb0a9ca79609767 WatchSource:0}: Error finding container e5d2c0f1bb1b02e4b28459e24b78633894076beb8b8bed393cb0a9ca79609767: Status 404 returned error can't find the container with id e5d2c0f1bb1b02e4b28459e24b78633894076beb8b8bed393cb0a9ca79609767
Sep 14 08:07:08.437430 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:08.437391    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq" event={"ID":"acf169ea-91c0-4869-be34-d9a3e39128d5","Type":"ContainerStarted","Data":"1c405cb6a3ac9c55abff7b1329947981fd8d4149490055b0f45065c24e93150c"}
Sep 14 08:07:08.437840 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:08.437457    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq" event={"ID":"acf169ea-91c0-4869-be34-d9a3e39128d5","Type":"ContainerStarted","Data":"e5d2c0f1bb1b02e4b28459e24b78633894076beb8b8bed393cb0a9ca79609767"}
Sep 14 08:07:08.454088 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:08.454040    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq" podStartSLOduration=1.4540244279999999 podStartE2EDuration="1.454024428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:08.453024042 +0000 UTC m=+1149.081911670" watchObservedRunningTime="2026-09-14 08:07:08.454024428 +0000 UTC m=+1149.082912055"
Sep 14 08:07:09.447194 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:09.447155    2563 generic.go:356] "Generic (PLEG): container finished" podID="acf169ea-91c0-4869-be34-d9a3e39128d5" containerID="1c405cb6a3ac9c55abff7b1329947981fd8d4149490055b0f45065c24e93150c" exitCode=0
Sep 14 08:07:09.447607 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:09.447238    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq" event={"ID":"acf169ea-91c0-4869-be34-d9a3e39128d5","Type":"ContainerDied","Data":"1c405cb6a3ac9c55abff7b1329947981fd8d4149490055b0f45065c24e93150c"}
Sep 14 08:07:10.600160 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:10.600134    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq"
Sep 14 08:07:10.666310 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:10.666278    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf169ea-91c0-4869-be34-d9a3e39128d5-maas-api-service-ca\") pod \"acf169ea-91c0-4869-be34-d9a3e39128d5\" (UID: \"acf169ea-91c0-4869-be34-d9a3e39128d5\") "
Sep 14 08:07:10.666602 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:10.666581    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf169ea-91c0-4869-be34-d9a3e39128d5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "acf169ea-91c0-4869-be34-d9a3e39128d5" (UID: "acf169ea-91c0-4869-be34-d9a3e39128d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:10.767618 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:10.767544    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf169ea-91c0-4869-be34-d9a3e39128d5-maas-api-service-ca\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:11.462009 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:11.461972    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq" event={"ID":"acf169ea-91c0-4869-be34-d9a3e39128d5","Type":"ContainerDied","Data":"e5d2c0f1bb1b02e4b28459e24b78633894076beb8b8bed393cb0a9ca79609767"}
Sep 14 08:07:11.462009 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:11.462000    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq"
Sep 14 08:07:11.462206 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:11.462020    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5d2c0f1bb1b02e4b28459e24b78633894076beb8b8bed393cb0a9ca79609767"
Sep 14 08:07:16.761236 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:16.761194    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"]
Sep 14 08:07:16.762022 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:16.761959    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw" podUID="739afdbf-b567-4d2f-9553-82749d1a7cf7" containerName="payload-processing" containerID="cri-o://bc2a0b4f14e919266205a4c890fb42974dc954ded9a21a81472f48b7f33398dc" gracePeriod=30
Sep 14 08:07:16.773397 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:16.773356    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"]
Sep 14 08:07:16.773812 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:16.773778    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr" podUID="7d998131-622d-4d96-808e-e6341bf0f52c" containerName="payload-pre-processing" containerID="cri-o://9b33c984930ded6ffadd335438eecf6d8fde35dec40859ac58989735758e4c81" gracePeriod=30
Sep 14 08:07:17.044136 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.044101    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"
Sep 14 08:07:17.077318 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.077290    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"
Sep 14 08:07:17.132222 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.132179    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmblc\" (UniqueName: \"kubernetes.io/projected/739afdbf-b567-4d2f-9553-82749d1a7cf7-kube-api-access-lmblc\") pod \"739afdbf-b567-4d2f-9553-82749d1a7cf7\" (UID: \"739afdbf-b567-4d2f-9553-82749d1a7cf7\") "
Sep 14 08:07:17.132222 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.132226    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d998131-622d-4d96-808e-e6341bf0f52c-plugins-config-volume\") pod \"7d998131-622d-4d96-808e-e6341bf0f52c\" (UID: \"7d998131-622d-4d96-808e-e6341bf0f52c\") "
Sep 14 08:07:17.132480 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.132253    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wntqj\" (UniqueName: \"kubernetes.io/projected/7d998131-622d-4d96-808e-e6341bf0f52c-kube-api-access-wntqj\") pod \"7d998131-622d-4d96-808e-e6341bf0f52c\" (UID: \"7d998131-622d-4d96-808e-e6341bf0f52c\") "
Sep 14 08:07:17.132480 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.132301    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/739afdbf-b567-4d2f-9553-82749d1a7cf7-plugins-config-volume\") pod \"739afdbf-b567-4d2f-9553-82749d1a7cf7\" (UID: \"739afdbf-b567-4d2f-9553-82749d1a7cf7\") "
Sep 14 08:07:17.132705 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.132680    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d998131-622d-4d96-808e-e6341bf0f52c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d998131-622d-4d96-808e-e6341bf0f52c" (UID: "7d998131-622d-4d96-808e-e6341bf0f52c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:17.132797 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.132748    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/739afdbf-b567-4d2f-9553-82749d1a7cf7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "739afdbf-b567-4d2f-9553-82749d1a7cf7" (UID: "739afdbf-b567-4d2f-9553-82749d1a7cf7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:17.134280 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.134256    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d998131-622d-4d96-808e-e6341bf0f52c-kube-api-access-wntqj" (OuterVolumeSpecName: "kube-api-access-wntqj") pod "7d998131-622d-4d96-808e-e6341bf0f52c" (UID: "7d998131-622d-4d96-808e-e6341bf0f52c"). InnerVolumeSpecName "kube-api-access-wntqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:17.134419 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.134329    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/739afdbf-b567-4d2f-9553-82749d1a7cf7-kube-api-access-lmblc" (OuterVolumeSpecName: "kube-api-access-lmblc") pod "739afdbf-b567-4d2f-9553-82749d1a7cf7" (UID: "739afdbf-b567-4d2f-9553-82749d1a7cf7"). InnerVolumeSpecName "kube-api-access-lmblc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:17.233497 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.233465    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmblc\" (UniqueName: \"kubernetes.io/projected/739afdbf-b567-4d2f-9553-82749d1a7cf7-kube-api-access-lmblc\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:17.233497 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.233494    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d998131-622d-4d96-808e-e6341bf0f52c-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:17.233497 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.233503    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wntqj\" (UniqueName: \"kubernetes.io/projected/7d998131-622d-4d96-808e-e6341bf0f52c-kube-api-access-wntqj\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:17.233805 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.233512    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/739afdbf-b567-4d2f-9553-82749d1a7cf7-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:17.505069 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.505035    2563 generic.go:356] "Generic (PLEG): container finished" podID="7d998131-622d-4d96-808e-e6341bf0f52c" containerID="9b33c984930ded6ffadd335438eecf6d8fde35dec40859ac58989735758e4c81" exitCode=0
Sep 14 08:07:17.505265 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.505080    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"
Sep 14 08:07:17.505265 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.505125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr" event={"ID":"7d998131-622d-4d96-808e-e6341bf0f52c","Type":"ContainerDied","Data":"9b33c984930ded6ffadd335438eecf6d8fde35dec40859ac58989735758e4c81"}
Sep 14 08:07:17.505265 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.505189    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr" event={"ID":"7d998131-622d-4d96-808e-e6341bf0f52c","Type":"ContainerDied","Data":"b87f488c092d36511061f969814bb31d3989d5b1cb0f6dd938b3fc6e7a0fd221"}
Sep 14 08:07:17.505265 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.505218    2563 scope.go:117] "RemoveContainer" containerID="9b33c984930ded6ffadd335438eecf6d8fde35dec40859ac58989735758e4c81"
Sep 14 08:07:17.506492 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.506470    2563 generic.go:356] "Generic (PLEG): container finished" podID="739afdbf-b567-4d2f-9553-82749d1a7cf7" containerID="bc2a0b4f14e919266205a4c890fb42974dc954ded9a21a81472f48b7f33398dc" exitCode=0
Sep 14 08:07:17.506627 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.506501    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw" event={"ID":"739afdbf-b567-4d2f-9553-82749d1a7cf7","Type":"ContainerDied","Data":"bc2a0b4f14e919266205a4c890fb42974dc954ded9a21a81472f48b7f33398dc"}
Sep 14 08:07:17.506627 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.506523    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"
Sep 14 08:07:17.506627 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.506549    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw" event={"ID":"739afdbf-b567-4d2f-9553-82749d1a7cf7","Type":"ContainerDied","Data":"a80e8fb14f816ac834e99b1edf8500c95eb39378dbf68a746c33569268c80587"}
Sep 14 08:07:17.515491 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.515473    2563 scope.go:117] "RemoveContainer" containerID="9b33c984930ded6ffadd335438eecf6d8fde35dec40859ac58989735758e4c81"
Sep 14 08:07:17.515801 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:07:17.515779    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b33c984930ded6ffadd335438eecf6d8fde35dec40859ac58989735758e4c81\": container with ID starting with 9b33c984930ded6ffadd335438eecf6d8fde35dec40859ac58989735758e4c81 not found: ID does not exist" containerID="9b33c984930ded6ffadd335438eecf6d8fde35dec40859ac58989735758e4c81"
Sep 14 08:07:17.515872 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.515810    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b33c984930ded6ffadd335438eecf6d8fde35dec40859ac58989735758e4c81"} err="failed to get container status \"9b33c984930ded6ffadd335438eecf6d8fde35dec40859ac58989735758e4c81\": rpc error: code = NotFound desc = could not find container \"9b33c984930ded6ffadd335438eecf6d8fde35dec40859ac58989735758e4c81\": container with ID starting with 9b33c984930ded6ffadd335438eecf6d8fde35dec40859ac58989735758e4c81 not found: ID does not exist"
Sep 14 08:07:17.515872 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.515827    2563 scope.go:117] "RemoveContainer" containerID="bc2a0b4f14e919266205a4c890fb42974dc954ded9a21a81472f48b7f33398dc"
Sep 14 08:07:17.524727 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.524708    2563 scope.go:117] "RemoveContainer" containerID="bc2a0b4f14e919266205a4c890fb42974dc954ded9a21a81472f48b7f33398dc"
Sep 14 08:07:17.524994 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:07:17.524960    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc2a0b4f14e919266205a4c890fb42974dc954ded9a21a81472f48b7f33398dc\": container with ID starting with bc2a0b4f14e919266205a4c890fb42974dc954ded9a21a81472f48b7f33398dc not found: ID does not exist" containerID="bc2a0b4f14e919266205a4c890fb42974dc954ded9a21a81472f48b7f33398dc"
Sep 14 08:07:17.525093 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.524992    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2a0b4f14e919266205a4c890fb42974dc954ded9a21a81472f48b7f33398dc"} err="failed to get container status \"bc2a0b4f14e919266205a4c890fb42974dc954ded9a21a81472f48b7f33398dc\": rpc error: code = NotFound desc = could not find container \"bc2a0b4f14e919266205a4c890fb42974dc954ded9a21a81472f48b7f33398dc\": container with ID starting with bc2a0b4f14e919266205a4c890fb42974dc954ded9a21a81472f48b7f33398dc not found: ID does not exist"
Sep 14 08:07:17.535929 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.535907    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"]
Sep 14 08:07:17.547565 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.547541    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-73bd4a-8565c95b54-qmdkr"]
Sep 14 08:07:17.567571 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.567545    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"]
Sep 14 08:07:17.599459 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.599429    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-73bd4a-79fcbdc45-bq9nw"]
Sep 14 08:07:17.887220 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.887191    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="739afdbf-b567-4d2f-9553-82749d1a7cf7" path="/var/lib/kubelet/pods/739afdbf-b567-4d2f-9553-82749d1a7cf7/volumes"
Sep 14 08:07:17.887563 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:17.887556    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d998131-622d-4d96-808e-e6341bf0f52c" path="/var/lib/kubelet/pods/7d998131-622d-4d96-808e-e6341bf0f52c/volumes"
Sep 14 08:07:18.092632 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.092598    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"]
Sep 14 08:07:18.092884 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.092857    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2" podUID="c4dc89e9-1855-4713-8915-b553d65aee4e" containerName="payload-processing" containerID="cri-o://b356ea9de3205c927d001e5bdde5b501bf4aca77ab5dd9eb081fe8aef662ca71" gracePeriod=30
Sep 14 08:07:18.100839 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.100815    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"]
Sep 14 08:07:18.101173 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.101146    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm" podUID="63b131c7-8203-411e-ae1e-bf139fe6a246" containerName="payload-pre-processing" containerID="cri-o://b8f0c7b55c2ebed36eaa5cec1685182d58cce2d4c2a2ec86689a62709dd4aa74" gracePeriod=30
Sep 14 08:07:18.369441 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.369413    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"
Sep 14 08:07:18.395218 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.395195    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"
Sep 14 08:07:18.448744 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.448699    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zh5c\" (UniqueName: \"kubernetes.io/projected/63b131c7-8203-411e-ae1e-bf139fe6a246-kube-api-access-4zh5c\") pod \"63b131c7-8203-411e-ae1e-bf139fe6a246\" (UID: \"63b131c7-8203-411e-ae1e-bf139fe6a246\") "
Sep 14 08:07:18.448744 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.448744    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mkg2\" (UniqueName: \"kubernetes.io/projected/c4dc89e9-1855-4713-8915-b553d65aee4e-kube-api-access-8mkg2\") pod \"c4dc89e9-1855-4713-8915-b553d65aee4e\" (UID: \"c4dc89e9-1855-4713-8915-b553d65aee4e\") "
Sep 14 08:07:18.448975 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.448782    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63b131c7-8203-411e-ae1e-bf139fe6a246-plugins-config-volume\") pod \"63b131c7-8203-411e-ae1e-bf139fe6a246\" (UID: \"63b131c7-8203-411e-ae1e-bf139fe6a246\") "
Sep 14 08:07:18.448975 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.448857    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4dc89e9-1855-4713-8915-b553d65aee4e-plugins-config-volume\") pod \"c4dc89e9-1855-4713-8915-b553d65aee4e\" (UID: \"c4dc89e9-1855-4713-8915-b553d65aee4e\") "
Sep 14 08:07:18.449188 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.449156    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63b131c7-8203-411e-ae1e-bf139fe6a246-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "63b131c7-8203-411e-ae1e-bf139fe6a246" (UID: "63b131c7-8203-411e-ae1e-bf139fe6a246"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:18.449315 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.449280    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4dc89e9-1855-4713-8915-b553d65aee4e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c4dc89e9-1855-4713-8915-b553d65aee4e" (UID: "c4dc89e9-1855-4713-8915-b553d65aee4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:18.451140 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.451114    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63b131c7-8203-411e-ae1e-bf139fe6a246-kube-api-access-4zh5c" (OuterVolumeSpecName: "kube-api-access-4zh5c") pod "63b131c7-8203-411e-ae1e-bf139fe6a246" (UID: "63b131c7-8203-411e-ae1e-bf139fe6a246"). InnerVolumeSpecName "kube-api-access-4zh5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:18.451228 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.451164    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4dc89e9-1855-4713-8915-b553d65aee4e-kube-api-access-8mkg2" (OuterVolumeSpecName: "kube-api-access-8mkg2") pod "c4dc89e9-1855-4713-8915-b553d65aee4e" (UID: "c4dc89e9-1855-4713-8915-b553d65aee4e"). InnerVolumeSpecName "kube-api-access-8mkg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:18.465397 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.465365    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq"]
Sep 14 08:07:18.469236 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.469217    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3a842e-q7srq"]
Sep 14 08:07:18.515307 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.515273    2563 generic.go:356] "Generic (PLEG): container finished" podID="63b131c7-8203-411e-ae1e-bf139fe6a246" containerID="b8f0c7b55c2ebed36eaa5cec1685182d58cce2d4c2a2ec86689a62709dd4aa74" exitCode=0
Sep 14 08:07:18.515470 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.515317    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"
Sep 14 08:07:18.515470 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.515367    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm" event={"ID":"63b131c7-8203-411e-ae1e-bf139fe6a246","Type":"ContainerDied","Data":"b8f0c7b55c2ebed36eaa5cec1685182d58cce2d4c2a2ec86689a62709dd4aa74"}
Sep 14 08:07:18.515470 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.515418    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm" event={"ID":"63b131c7-8203-411e-ae1e-bf139fe6a246","Type":"ContainerDied","Data":"33e92aaa1601984eafb93029fad77611811237adbdf33f263c09e4c43d1915a0"}
Sep 14 08:07:18.515470 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.515449    2563 scope.go:117] "RemoveContainer" containerID="b8f0c7b55c2ebed36eaa5cec1685182d58cce2d4c2a2ec86689a62709dd4aa74"
Sep 14 08:07:18.516881 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.516860    2563 generic.go:356] "Generic (PLEG): container finished" podID="c4dc89e9-1855-4713-8915-b553d65aee4e" containerID="b356ea9de3205c927d001e5bdde5b501bf4aca77ab5dd9eb081fe8aef662ca71" exitCode=0
Sep 14 08:07:18.517004 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.516914    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"
Sep 14 08:07:18.517004 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.516935    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2" event={"ID":"c4dc89e9-1855-4713-8915-b553d65aee4e","Type":"ContainerDied","Data":"b356ea9de3205c927d001e5bdde5b501bf4aca77ab5dd9eb081fe8aef662ca71"}
Sep 14 08:07:18.517121 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.517008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2" event={"ID":"c4dc89e9-1855-4713-8915-b553d65aee4e","Type":"ContainerDied","Data":"f27d42011269741619b039072a16d62882cf75c72096741fd4f20db763871ae8"}
Sep 14 08:07:18.527032 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.527013    2563 scope.go:117] "RemoveContainer" containerID="b8f0c7b55c2ebed36eaa5cec1685182d58cce2d4c2a2ec86689a62709dd4aa74"
Sep 14 08:07:18.527304 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:07:18.527281    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8f0c7b55c2ebed36eaa5cec1685182d58cce2d4c2a2ec86689a62709dd4aa74\": container with ID starting with b8f0c7b55c2ebed36eaa5cec1685182d58cce2d4c2a2ec86689a62709dd4aa74 not found: ID does not exist" containerID="b8f0c7b55c2ebed36eaa5cec1685182d58cce2d4c2a2ec86689a62709dd4aa74"
Sep 14 08:07:18.527387 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.527312    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8f0c7b55c2ebed36eaa5cec1685182d58cce2d4c2a2ec86689a62709dd4aa74"} err="failed to get container status \"b8f0c7b55c2ebed36eaa5cec1685182d58cce2d4c2a2ec86689a62709dd4aa74\": rpc error: code = NotFound desc = could not find container \"b8f0c7b55c2ebed36eaa5cec1685182d58cce2d4c2a2ec86689a62709dd4aa74\": container with ID starting with b8f0c7b55c2ebed36eaa5cec1685182d58cce2d4c2a2ec86689a62709dd4aa74 not found: ID does not exist"
Sep 14 08:07:18.527387 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.527330    2563 scope.go:117] "RemoveContainer" containerID="b356ea9de3205c927d001e5bdde5b501bf4aca77ab5dd9eb081fe8aef662ca71"
Sep 14 08:07:18.536101 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.536084    2563 scope.go:117] "RemoveContainer" containerID="b356ea9de3205c927d001e5bdde5b501bf4aca77ab5dd9eb081fe8aef662ca71"
Sep 14 08:07:18.536401 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:07:18.536376    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b356ea9de3205c927d001e5bdde5b501bf4aca77ab5dd9eb081fe8aef662ca71\": container with ID starting with b356ea9de3205c927d001e5bdde5b501bf4aca77ab5dd9eb081fe8aef662ca71 not found: ID does not exist" containerID="b356ea9de3205c927d001e5bdde5b501bf4aca77ab5dd9eb081fe8aef662ca71"
Sep 14 08:07:18.536462 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.536411    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b356ea9de3205c927d001e5bdde5b501bf4aca77ab5dd9eb081fe8aef662ca71"} err="failed to get container status \"b356ea9de3205c927d001e5bdde5b501bf4aca77ab5dd9eb081fe8aef662ca71\": rpc error: code = NotFound desc = could not find container \"b356ea9de3205c927d001e5bdde5b501bf4aca77ab5dd9eb081fe8aef662ca71\": container with ID starting with b356ea9de3205c927d001e5bdde5b501bf4aca77ab5dd9eb081fe8aef662ca71 not found: ID does not exist"
Sep 14 08:07:18.544259 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.544238    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"]
Sep 14 08:07:18.550373 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.550342    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a842e-787d8795b6-xx2jm"]
Sep 14 08:07:18.550711 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.550693    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zh5c\" (UniqueName: \"kubernetes.io/projected/63b131c7-8203-411e-ae1e-bf139fe6a246-kube-api-access-4zh5c\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:18.550784 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.550713    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mkg2\" (UniqueName: \"kubernetes.io/projected/c4dc89e9-1855-4713-8915-b553d65aee4e-kube-api-access-8mkg2\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:18.550784 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.550727    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63b131c7-8203-411e-ae1e-bf139fe6a246-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:18.550784 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.550739    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4dc89e9-1855-4713-8915-b553d65aee4e-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:18.562822 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.562798    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"]
Sep 14 08:07:18.566253 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:18.566234    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3a842e-bb8b7db9f-zn4x2"]
Sep 14 08:07:19.888017 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:19.887985    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63b131c7-8203-411e-ae1e-bf139fe6a246" path="/var/lib/kubelet/pods/63b131c7-8203-411e-ae1e-bf139fe6a246/volumes"
Sep 14 08:07:19.888372 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:19.888311    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acf169ea-91c0-4869-be34-d9a3e39128d5" path="/var/lib/kubelet/pods/acf169ea-91c0-4869-be34-d9a3e39128d5/volumes"
Sep 14 08:07:19.888736 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:19.888717    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4dc89e9-1855-4713-8915-b553d65aee4e" path="/var/lib/kubelet/pods/c4dc89e9-1855-4713-8915-b553d65aee4e/volumes"
Sep 14 08:07:26.539169 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539089    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct"]
Sep 14 08:07:26.539740 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539723    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63b131c7-8203-411e-ae1e-bf139fe6a246" containerName="payload-pre-processing"
Sep 14 08:07:26.539793 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539743    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="63b131c7-8203-411e-ae1e-bf139fe6a246" containerName="payload-pre-processing"
Sep 14 08:07:26.539793 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539767    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="739afdbf-b567-4d2f-9553-82749d1a7cf7" containerName="payload-processing"
Sep 14 08:07:26.539793 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539776    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="739afdbf-b567-4d2f-9553-82749d1a7cf7" containerName="payload-processing"
Sep 14 08:07:26.539793 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539789    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4dc89e9-1855-4713-8915-b553d65aee4e" containerName="payload-processing"
Sep 14 08:07:26.539920 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539798    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4dc89e9-1855-4713-8915-b553d65aee4e" containerName="payload-processing"
Sep 14 08:07:26.539920 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539809    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d998131-622d-4d96-808e-e6341bf0f52c" containerName="payload-pre-processing"
Sep 14 08:07:26.539920 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539818    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d998131-622d-4d96-808e-e6341bf0f52c" containerName="payload-pre-processing"
Sep 14 08:07:26.539920 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539838    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf169ea-91c0-4869-be34-d9a3e39128d5" containerName="revoke-keys"
Sep 14 08:07:26.539920 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539847    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf169ea-91c0-4869-be34-d9a3e39128d5" containerName="revoke-keys"
Sep 14 08:07:26.540066 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539943    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="63b131c7-8203-411e-ae1e-bf139fe6a246" containerName="payload-pre-processing"
Sep 14 08:07:26.540066 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539955    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d998131-622d-4d96-808e-e6341bf0f52c" containerName="payload-pre-processing"
Sep 14 08:07:26.540066 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539967    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="739afdbf-b567-4d2f-9553-82749d1a7cf7" containerName="payload-processing"
Sep 14 08:07:26.540066 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539976    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="acf169ea-91c0-4869-be34-d9a3e39128d5" containerName="revoke-keys"
Sep 14 08:07:26.540066 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.539988    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4dc89e9-1855-4713-8915-b553d65aee4e" containerName="payload-processing"
Sep 14 08:07:26.545106 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.545085    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct"
Sep 14 08:07:26.547220 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.547203    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:07:26.550141 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.550117    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct"]
Sep 14 08:07:26.623820 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.623790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e9a9834-e676-465c-9d92-3be29209396a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct\" (UID: \"1e9a9834-e676-465c-9d92-3be29209396a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct"
Sep 14 08:07:26.724805 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.724767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e9a9834-e676-465c-9d92-3be29209396a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct\" (UID: \"1e9a9834-e676-465c-9d92-3be29209396a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct"
Sep 14 08:07:26.725351 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.725331    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e9a9834-e676-465c-9d92-3be29209396a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct\" (UID: \"1e9a9834-e676-465c-9d92-3be29209396a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct"
Sep 14 08:07:26.859563 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:26.859524    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct"
Sep 14 08:07:27.132433 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:27.132402    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct"]
Sep 14 08:07:27.133610 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:07:27.133581    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e9a9834_e676_465c_9d92_3be29209396a.slice/crio-b0ea31edf19ff0b2bbc0d17fc11f90844a0e5f940d8437eda86035332895ae8a WatchSource:0}: Error finding container b0ea31edf19ff0b2bbc0d17fc11f90844a0e5f940d8437eda86035332895ae8a: Status 404 returned error can't find the container with id b0ea31edf19ff0b2bbc0d17fc11f90844a0e5f940d8437eda86035332895ae8a
Sep 14 08:07:27.584834 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:27.584795    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct" event={"ID":"1e9a9834-e676-465c-9d92-3be29209396a","Type":"ContainerStarted","Data":"e46a55a9898c40b57846fa3cbfd9429497d1ef56b0e3ae1e8ac3cf833966d0f8"}
Sep 14 08:07:27.585293 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:27.584848    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct" event={"ID":"1e9a9834-e676-465c-9d92-3be29209396a","Type":"ContainerStarted","Data":"b0ea31edf19ff0b2bbc0d17fc11f90844a0e5f940d8437eda86035332895ae8a"}
Sep 14 08:07:27.602351 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:27.602308    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct" podStartSLOduration=1.602289123 podStartE2EDuration="1.602289123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:27.599165775 +0000 UTC m=+1168.228053408" watchObservedRunningTime="2026-09-14 08:07:27.602289123 +0000 UTC m=+1168.231176776"
Sep 14 08:07:28.558402 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.558370    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"]
Sep 14 08:07:28.561993 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.561975    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"
Sep 14 08:07:28.564240 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.564217    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-8b87debd-dockercfg-s5dgc\""
Sep 14 08:07:28.564328 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.564253    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-8b87debd\""
Sep 14 08:07:28.572016 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.571996    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"]
Sep 14 08:07:28.591273 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.591250    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"]
Sep 14 08:07:28.594279 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.594242    2563 generic.go:356] "Generic (PLEG): container finished" podID="1e9a9834-e676-465c-9d92-3be29209396a" containerID="e46a55a9898c40b57846fa3cbfd9429497d1ef56b0e3ae1e8ac3cf833966d0f8" exitCode=0
Sep 14 08:07:28.602932 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.602901    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct" event={"ID":"1e9a9834-e676-465c-9d92-3be29209396a","Type":"ContainerDied","Data":"e46a55a9898c40b57846fa3cbfd9429497d1ef56b0e3ae1e8ac3cf833966d0f8"}
Sep 14 08:07:28.603048 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.603023    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"
Sep 14 08:07:28.604758 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.604727    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"]
Sep 14 08:07:28.642315 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.642280    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh7r9\" (UniqueName: \"kubernetes.io/projected/585eb966-1593-47d7-af00-e8cdb909a731-kube-api-access-rh7r9\") pod \"payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs\" (UID: \"585eb966-1593-47d7-af00-e8cdb909a731\") " pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"
Sep 14 08:07:28.642710 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.642678    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/585eb966-1593-47d7-af00-e8cdb909a731-plugins-config-volume\") pod \"payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs\" (UID: \"585eb966-1593-47d7-af00-e8cdb909a731\") " pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"
Sep 14 08:07:28.743845 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.743799    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp6w8\" (UniqueName: \"kubernetes.io/projected/de595a89-111e-410c-a56c-a65a5144cce8-kube-api-access-jp6w8\") pod \"payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl\" (UID: \"de595a89-111e-410c-a56c-a65a5144cce8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"
Sep 14 08:07:28.744027 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.743854    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/585eb966-1593-47d7-af00-e8cdb909a731-plugins-config-volume\") pod \"payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs\" (UID: \"585eb966-1593-47d7-af00-e8cdb909a731\") " pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"
Sep 14 08:07:28.744027 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.743955    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh7r9\" (UniqueName: \"kubernetes.io/projected/585eb966-1593-47d7-af00-e8cdb909a731-kube-api-access-rh7r9\") pod \"payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs\" (UID: \"585eb966-1593-47d7-af00-e8cdb909a731\") " pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"
Sep 14 08:07:28.744027 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.744000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de595a89-111e-410c-a56c-a65a5144cce8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl\" (UID: \"de595a89-111e-410c-a56c-a65a5144cce8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"
Sep 14 08:07:28.744703 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.744679    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/585eb966-1593-47d7-af00-e8cdb909a731-plugins-config-volume\") pod \"payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs\" (UID: \"585eb966-1593-47d7-af00-e8cdb909a731\") " pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"
Sep 14 08:07:28.752193 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.752169    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh7r9\" (UniqueName: \"kubernetes.io/projected/585eb966-1593-47d7-af00-e8cdb909a731-kube-api-access-rh7r9\") pod \"payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs\" (UID: \"585eb966-1593-47d7-af00-e8cdb909a731\") " pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"
Sep 14 08:07:28.844791 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.844756    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp6w8\" (UniqueName: \"kubernetes.io/projected/de595a89-111e-410c-a56c-a65a5144cce8-kube-api-access-jp6w8\") pod \"payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl\" (UID: \"de595a89-111e-410c-a56c-a65a5144cce8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"
Sep 14 08:07:28.844949 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.844854    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de595a89-111e-410c-a56c-a65a5144cce8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl\" (UID: \"de595a89-111e-410c-a56c-a65a5144cce8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"
Sep 14 08:07:28.845559 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.845535    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de595a89-111e-410c-a56c-a65a5144cce8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl\" (UID: \"de595a89-111e-410c-a56c-a65a5144cce8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"
Sep 14 08:07:28.852600 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.852578    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp6w8\" (UniqueName: \"kubernetes.io/projected/de595a89-111e-410c-a56c-a65a5144cce8-kube-api-access-jp6w8\") pod \"payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl\" (UID: \"de595a89-111e-410c-a56c-a65a5144cce8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"
Sep 14 08:07:28.873569 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.873546    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"
Sep 14 08:07:28.914892 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:28.914419    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"
Sep 14 08:07:29.149998 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.149969    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"]
Sep 14 08:07:29.150372 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:07:29.150347    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod585eb966_1593_47d7_af00_e8cdb909a731.slice/crio-2c4888e39bdb003d71776fc9c6c7bfaf84fdb50c1b164950a9a4d6f9333d48ba WatchSource:0}: Error finding container 2c4888e39bdb003d71776fc9c6c7bfaf84fdb50c1b164950a9a4d6f9333d48ba: Status 404 returned error can't find the container with id 2c4888e39bdb003d71776fc9c6c7bfaf84fdb50c1b164950a9a4d6f9333d48ba
Sep 14 08:07:29.197261 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:07:29.197222    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde595a89_111e_410c_a56c_a65a5144cce8.slice/crio-54d409b3e0a3a2bbf20a93a539dca9b57476de8ea2c03272490723d2ffdac8f2 WatchSource:0}: Error finding container 54d409b3e0a3a2bbf20a93a539dca9b57476de8ea2c03272490723d2ffdac8f2: Status 404 returned error can't find the container with id 54d409b3e0a3a2bbf20a93a539dca9b57476de8ea2c03272490723d2ffdac8f2
Sep 14 08:07:29.197448 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.197423    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"]
Sep 14 08:07:29.603135 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.603097    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl" event={"ID":"de595a89-111e-410c-a56c-a65a5144cce8","Type":"ContainerStarted","Data":"90a731e62118371e8ffa7b26d4990d79b6795703a2fdea03aeea6022cf4ded55"}
Sep 14 08:07:29.603599 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.603155    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl" event={"ID":"de595a89-111e-410c-a56c-a65a5144cce8","Type":"ContainerStarted","Data":"54d409b3e0a3a2bbf20a93a539dca9b57476de8ea2c03272490723d2ffdac8f2"}
Sep 14 08:07:29.603599 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.603218    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"
Sep 14 08:07:29.605068 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.605044    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs" event={"ID":"585eb966-1593-47d7-af00-e8cdb909a731","Type":"ContainerStarted","Data":"7a75bac4444c9cbcfa0aada312d23390b616166cb45f86f0df3d7e0def5b3fab"}
Sep 14 08:07:29.605177 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.605091    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"
Sep 14 08:07:29.605177 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.605113    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs" event={"ID":"585eb966-1593-47d7-af00-e8cdb909a731","Type":"ContainerStarted","Data":"2c4888e39bdb003d71776fc9c6c7bfaf84fdb50c1b164950a9a4d6f9333d48ba"}
Sep 14 08:07:29.620694 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.620654    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl" podStartSLOduration=1.6206227210000002 podStartE2EDuration="1.620622721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:29.618533851 +0000 UTC m=+1170.247421508" watchObservedRunningTime="2026-09-14 08:07:29.620622721 +0000 UTC m=+1170.249510347"
Sep 14 08:07:29.636098 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.636026    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs" podStartSLOduration=1.636009528 podStartE2EDuration="1.636009528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:29.632888895 +0000 UTC m=+1170.261776546" watchObservedRunningTime="2026-09-14 08:07:29.636009528 +0000 UTC m=+1170.264897155"
Sep 14 08:07:29.748664 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.748605    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct"
Sep 14 08:07:29.858137 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.858058    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e9a9834-e676-465c-9d92-3be29209396a-maas-api-service-ca\") pod \"1e9a9834-e676-465c-9d92-3be29209396a\" (UID: \"1e9a9834-e676-465c-9d92-3be29209396a\") "
Sep 14 08:07:29.858437 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.858413    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e9a9834-e676-465c-9d92-3be29209396a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1e9a9834-e676-465c-9d92-3be29209396a" (UID: "1e9a9834-e676-465c-9d92-3be29209396a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:29.959173 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:29.959146    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e9a9834-e676-465c-9d92-3be29209396a-maas-api-service-ca\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:30.614138 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:30.614110    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct"
Sep 14 08:07:30.614533 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:30.614107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct" event={"ID":"1e9a9834-e676-465c-9d92-3be29209396a","Type":"ContainerDied","Data":"b0ea31edf19ff0b2bbc0d17fc11f90844a0e5f940d8437eda86035332895ae8a"}
Sep 14 08:07:30.614533 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:30.614233    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0ea31edf19ff0b2bbc0d17fc11f90844a0e5f940d8437eda86035332895ae8a"
Sep 14 08:07:37.494500 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.494458    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"]
Sep 14 08:07:37.494976 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.494771    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd" podUID="ae6dcda5-b5d5-454c-aefd-69b2457491b3" containerName="payload-processing" containerID="cri-o://15bbd0bc413ae3b58052af72728496feaeae0940c7c4edd934224d15a82a31dc" gracePeriod=30
Sep 14 08:07:37.499112 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.499084    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"]
Sep 14 08:07:37.499503 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.499451    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7" podUID="95f9ae36-17dc-4c6d-9909-2bf2970d977f" containerName="payload-pre-processing" containerID="cri-o://ee2153af38e16907134ac3e154dc4416dcea3668167376ac321821f066eb1d4b" gracePeriod=30
Sep 14 08:07:37.659605 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.659557    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct"]
Sep 14 08:07:37.663658 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.663614    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ae4e6-qckct"]
Sep 14 08:07:37.664751 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.664726    2563 generic.go:356] "Generic (PLEG): container finished" podID="ae6dcda5-b5d5-454c-aefd-69b2457491b3" containerID="15bbd0bc413ae3b58052af72728496feaeae0940c7c4edd934224d15a82a31dc" exitCode=0
Sep 14 08:07:37.664868 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.664792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd" event={"ID":"ae6dcda5-b5d5-454c-aefd-69b2457491b3","Type":"ContainerDied","Data":"15bbd0bc413ae3b58052af72728496feaeae0940c7c4edd934224d15a82a31dc"}
Sep 14 08:07:37.667185 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.667163    2563 generic.go:356] "Generic (PLEG): container finished" podID="95f9ae36-17dc-4c6d-9909-2bf2970d977f" containerID="ee2153af38e16907134ac3e154dc4416dcea3668167376ac321821f066eb1d4b" exitCode=0
Sep 14 08:07:37.667375 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.667235    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7" event={"ID":"95f9ae36-17dc-4c6d-9909-2bf2970d977f","Type":"ContainerDied","Data":"ee2153af38e16907134ac3e154dc4416dcea3668167376ac321821f066eb1d4b"}
Sep 14 08:07:37.793336 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.793313    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"
Sep 14 08:07:37.796232 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.796207    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"
Sep 14 08:07:37.886679 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.886618    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e9a9834-e676-465c-9d92-3be29209396a" path="/var/lib/kubelet/pods/1e9a9834-e676-465c-9d92-3be29209396a/volumes"
Sep 14 08:07:37.936363 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.936341    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hc48\" (UniqueName: \"kubernetes.io/projected/95f9ae36-17dc-4c6d-9909-2bf2970d977f-kube-api-access-5hc48\") pod \"95f9ae36-17dc-4c6d-9909-2bf2970d977f\" (UID: \"95f9ae36-17dc-4c6d-9909-2bf2970d977f\") "
Sep 14 08:07:37.936510 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.936379    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae6dcda5-b5d5-454c-aefd-69b2457491b3-plugins-config-volume\") pod \"ae6dcda5-b5d5-454c-aefd-69b2457491b3\" (UID: \"ae6dcda5-b5d5-454c-aefd-69b2457491b3\") "
Sep 14 08:07:37.936510 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.936452    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95f9ae36-17dc-4c6d-9909-2bf2970d977f-plugins-config-volume\") pod \"95f9ae36-17dc-4c6d-9909-2bf2970d977f\" (UID: \"95f9ae36-17dc-4c6d-9909-2bf2970d977f\") "
Sep 14 08:07:37.936510 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.936503    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-br4fb\" (UniqueName: \"kubernetes.io/projected/ae6dcda5-b5d5-454c-aefd-69b2457491b3-kube-api-access-br4fb\") pod \"ae6dcda5-b5d5-454c-aefd-69b2457491b3\" (UID: \"ae6dcda5-b5d5-454c-aefd-69b2457491b3\") "
Sep 14 08:07:37.936844 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.936818    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95f9ae36-17dc-4c6d-9909-2bf2970d977f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "95f9ae36-17dc-4c6d-9909-2bf2970d977f" (UID: "95f9ae36-17dc-4c6d-9909-2bf2970d977f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:37.936844 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.936839    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae6dcda5-b5d5-454c-aefd-69b2457491b3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ae6dcda5-b5d5-454c-aefd-69b2457491b3" (UID: "ae6dcda5-b5d5-454c-aefd-69b2457491b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:37.938503 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.938480    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95f9ae36-17dc-4c6d-9909-2bf2970d977f-kube-api-access-5hc48" (OuterVolumeSpecName: "kube-api-access-5hc48") pod "95f9ae36-17dc-4c6d-9909-2bf2970d977f" (UID: "95f9ae36-17dc-4c6d-9909-2bf2970d977f"). InnerVolumeSpecName "kube-api-access-5hc48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:37.938623 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:37.938566    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae6dcda5-b5d5-454c-aefd-69b2457491b3-kube-api-access-br4fb" (OuterVolumeSpecName: "kube-api-access-br4fb") pod "ae6dcda5-b5d5-454c-aefd-69b2457491b3" (UID: "ae6dcda5-b5d5-454c-aefd-69b2457491b3"). InnerVolumeSpecName "kube-api-access-br4fb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:38.037969 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.037900    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-br4fb\" (UniqueName: \"kubernetes.io/projected/ae6dcda5-b5d5-454c-aefd-69b2457491b3-kube-api-access-br4fb\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:38.037969 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.037925    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hc48\" (UniqueName: \"kubernetes.io/projected/95f9ae36-17dc-4c6d-9909-2bf2970d977f-kube-api-access-5hc48\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:38.037969 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.037934    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae6dcda5-b5d5-454c-aefd-69b2457491b3-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:38.037969 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.037943    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95f9ae36-17dc-4c6d-9909-2bf2970d977f-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:07:38.675514 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.675488    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"
Sep 14 08:07:38.676036 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.675520    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd" event={"ID":"ae6dcda5-b5d5-454c-aefd-69b2457491b3","Type":"ContainerDied","Data":"943820aad6fa9b857a6ef84bf7e8b9b41deeab431ad366bc0059a524a426a855"}
Sep 14 08:07:38.676036 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.675578    2563 scope.go:117] "RemoveContainer" containerID="15bbd0bc413ae3b58052af72728496feaeae0940c7c4edd934224d15a82a31dc"
Sep 14 08:07:38.677415 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.677398    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7" event={"ID":"95f9ae36-17dc-4c6d-9909-2bf2970d977f","Type":"ContainerDied","Data":"6202a72e04c3193a4304e015c05128007ef84cebb3900a387fa45a6c8419522c"}
Sep 14 08:07:38.677486 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.677424    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"
Sep 14 08:07:38.686594 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.686432    2563 scope.go:117] "RemoveContainer" containerID="ee2153af38e16907134ac3e154dc4416dcea3668167376ac321821f066eb1d4b"
Sep 14 08:07:38.703684 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.703619    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"]
Sep 14 08:07:38.706286 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.706263    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5ae4e6-7bf8586cc8-jhrp7"]
Sep 14 08:07:38.718477 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.718458    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"]
Sep 14 08:07:38.721664 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:38.721629    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5ae4e6-755f868579-r5wsd"]
Sep 14 08:07:39.887448 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:39.887414    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95f9ae36-17dc-4c6d-9909-2bf2970d977f" path="/var/lib/kubelet/pods/95f9ae36-17dc-4c6d-9909-2bf2970d977f/volumes"
Sep 14 08:07:39.887848 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:39.887796    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae6dcda5-b5d5-454c-aefd-69b2457491b3" path="/var/lib/kubelet/pods/ae6dcda5-b5d5-454c-aefd-69b2457491b3/volumes"
Sep 14 08:07:40.617831 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:40.617798    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"
Sep 14 08:07:40.619061 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:40.618066    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"
Sep 14 08:07:53.228795 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.228766    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"]
Sep 14 08:07:53.229195 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.229183    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e9a9834-e676-465c-9d92-3be29209396a" containerName="revoke-keys"
Sep 14 08:07:53.229252 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.229197    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e9a9834-e676-465c-9d92-3be29209396a" containerName="revoke-keys"
Sep 14 08:07:53.229252 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.229214    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae6dcda5-b5d5-454c-aefd-69b2457491b3" containerName="payload-processing"
Sep 14 08:07:53.229252 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.229219    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae6dcda5-b5d5-454c-aefd-69b2457491b3" containerName="payload-processing"
Sep 14 08:07:53.229252 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.229226    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95f9ae36-17dc-4c6d-9909-2bf2970d977f" containerName="payload-pre-processing"
Sep 14 08:07:53.229252 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.229231    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="95f9ae36-17dc-4c6d-9909-2bf2970d977f" containerName="payload-pre-processing"
Sep 14 08:07:53.229423 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.229298    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e9a9834-e676-465c-9d92-3be29209396a" containerName="revoke-keys"
Sep 14 08:07:53.229423 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.229307    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="95f9ae36-17dc-4c6d-9909-2bf2970d977f" containerName="payload-pre-processing"
Sep 14 08:07:53.229423 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.229315    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae6dcda5-b5d5-454c-aefd-69b2457491b3" containerName="payload-processing"
Sep 14 08:07:53.234230 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.234212    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"
Sep 14 08:07:53.237428 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.237406    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-7746e4\""
Sep 14 08:07:53.254341 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.254311    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"]
Sep 14 08:07:53.273614 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.273588    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"]
Sep 14 08:07:53.273614 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.273622    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"]
Sep 14 08:07:53.273846 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.273705    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"
Sep 14 08:07:53.278512 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.278492    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-7746e4-dockercfg-8qlsm\""
Sep 14 08:07:53.280453 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.280428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2252k\" (UniqueName: \"kubernetes.io/projected/e2c5c0e1-e434-493e-be41-f8e966a03648-kube-api-access-2252k\") pod \"payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld\" (UID: \"e2c5c0e1-e434-493e-be41-f8e966a03648\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"
Sep 14 08:07:53.280576 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.280473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2c5c0e1-e434-493e-be41-f8e966a03648-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld\" (UID: \"e2c5c0e1-e434-493e-be41-f8e966a03648\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"
Sep 14 08:07:53.381478 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.381445    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2252k\" (UniqueName: \"kubernetes.io/projected/e2c5c0e1-e434-493e-be41-f8e966a03648-kube-api-access-2252k\") pod \"payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld\" (UID: \"e2c5c0e1-e434-493e-be41-f8e966a03648\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"
Sep 14 08:07:53.381478 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.381482    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z9g8\" (UniqueName: \"kubernetes.io/projected/1e91a6a2-9aaa-47b8-9960-b417ba260f7a-kube-api-access-7z9g8\") pod \"payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d\" (UID: \"1e91a6a2-9aaa-47b8-9960-b417ba260f7a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"
Sep 14 08:07:53.381710 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.381505    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2c5c0e1-e434-493e-be41-f8e966a03648-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld\" (UID: \"e2c5c0e1-e434-493e-be41-f8e966a03648\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"
Sep 14 08:07:53.381710 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.381614    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e91a6a2-9aaa-47b8-9960-b417ba260f7a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d\" (UID: \"1e91a6a2-9aaa-47b8-9960-b417ba260f7a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"
Sep 14 08:07:53.382229 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.382210    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2c5c0e1-e434-493e-be41-f8e966a03648-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld\" (UID: \"e2c5c0e1-e434-493e-be41-f8e966a03648\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"
Sep 14 08:07:53.390596 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.390567    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2252k\" (UniqueName: \"kubernetes.io/projected/e2c5c0e1-e434-493e-be41-f8e966a03648-kube-api-access-2252k\") pod \"payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld\" (UID: \"e2c5c0e1-e434-493e-be41-f8e966a03648\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"
Sep 14 08:07:53.483036 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.482964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e91a6a2-9aaa-47b8-9960-b417ba260f7a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d\" (UID: \"1e91a6a2-9aaa-47b8-9960-b417ba260f7a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"
Sep 14 08:07:53.483175 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.483039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z9g8\" (UniqueName: \"kubernetes.io/projected/1e91a6a2-9aaa-47b8-9960-b417ba260f7a-kube-api-access-7z9g8\") pod \"payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d\" (UID: \"1e91a6a2-9aaa-47b8-9960-b417ba260f7a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"
Sep 14 08:07:53.483676 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.483627    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e91a6a2-9aaa-47b8-9960-b417ba260f7a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d\" (UID: \"1e91a6a2-9aaa-47b8-9960-b417ba260f7a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"
Sep 14 08:07:53.492250 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.492233    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z9g8\" (UniqueName: \"kubernetes.io/projected/1e91a6a2-9aaa-47b8-9960-b417ba260f7a-kube-api-access-7z9g8\") pod \"payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d\" (UID: \"1e91a6a2-9aaa-47b8-9960-b417ba260f7a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"
Sep 14 08:07:53.548938 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.548902    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"
Sep 14 08:07:53.584330 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.584301    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"
Sep 14 08:07:53.832031 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.832002    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"]
Sep 14 08:07:53.832347 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:07:53.832327    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2c5c0e1_e434_493e_be41_f8e966a03648.slice/crio-01bc3a687f6ab505814fc5c6fe86941b09694b93e90a92df45ff76356d8e9f60 WatchSource:0}: Error finding container 01bc3a687f6ab505814fc5c6fe86941b09694b93e90a92df45ff76356d8e9f60: Status 404 returned error can't find the container with id 01bc3a687f6ab505814fc5c6fe86941b09694b93e90a92df45ff76356d8e9f60
Sep 14 08:07:53.874971 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:53.874942    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"]
Sep 14 08:07:53.875896 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:07:53.875868    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e91a6a2_9aaa_47b8_9960_b417ba260f7a.slice/crio-05b7107dadf76a82acbea1bd01c97fb30f891065a0cdab990a3e07dae03733ec WatchSource:0}: Error finding container 05b7107dadf76a82acbea1bd01c97fb30f891065a0cdab990a3e07dae03733ec: Status 404 returned error can't find the container with id 05b7107dadf76a82acbea1bd01c97fb30f891065a0cdab990a3e07dae03733ec
Sep 14 08:07:54.788652 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:54.788612    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld" event={"ID":"e2c5c0e1-e434-493e-be41-f8e966a03648","Type":"ContainerStarted","Data":"99278a18a0bd895d9cea225f2f9353da8c09695be15d58373e7f5f485d7f30a8"}
Sep 14 08:07:54.789073 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:54.788698    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld" event={"ID":"e2c5c0e1-e434-493e-be41-f8e966a03648","Type":"ContainerStarted","Data":"01bc3a687f6ab505814fc5c6fe86941b09694b93e90a92df45ff76356d8e9f60"}
Sep 14 08:07:54.789073 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:54.788727    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"
Sep 14 08:07:54.790445 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:54.790420    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d" event={"ID":"1e91a6a2-9aaa-47b8-9960-b417ba260f7a","Type":"ContainerStarted","Data":"38676bf04b758469e04fb9a58ad4bc6ba9e81a4a8e7d13afd7130a96d74d5c92"}
Sep 14 08:07:54.790594 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:54.790460    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d" event={"ID":"1e91a6a2-9aaa-47b8-9960-b417ba260f7a","Type":"ContainerStarted","Data":"05b7107dadf76a82acbea1bd01c97fb30f891065a0cdab990a3e07dae03733ec"}
Sep 14 08:07:54.790594 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:54.790546    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"
Sep 14 08:07:54.808222 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:54.808177    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld" podStartSLOduration=1.808164121 podStartE2EDuration="1.808164121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:54.805358815 +0000 UTC m=+1195.434246442" watchObservedRunningTime="2026-09-14 08:07:54.808164121 +0000 UTC m=+1195.437051747"
Sep 14 08:07:54.823979 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:54.823935    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d" podStartSLOduration=1.8239221799999998 podStartE2EDuration="1.82392218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:54.819836478 +0000 UTC m=+1195.448724132" watchObservedRunningTime="2026-09-14 08:07:54.82392218 +0000 UTC m=+1195.452809807"
Sep 14 08:07:59.887470 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:59.887438    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 08:07:59.890868 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:59.890847    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 08:07:59.995356 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:07:59.995330    2563 scope.go:117] "RemoveContainer" containerID="782916ecb8af7ee1ac23391b50cf45fa26f53f61e301333125e817244bed4adc"
Sep 14 08:08:05.798269 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:05.798234    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"
Sep 14 08:08:05.798677 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:05.798418    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"
Sep 14 08:08:28.146726 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.146690    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"]
Sep 14 08:08:28.150230 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.150212    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"
Sep 14 08:08:28.152633 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.152613    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-ea9d35-dockercfg-kjqn8\""
Sep 14 08:08:28.153072 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.152892    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-ea9d35\""
Sep 14 08:08:28.161688 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.161619    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"]
Sep 14 08:08:28.184599 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.184571    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"]
Sep 14 08:08:28.188190 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.188175    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"
Sep 14 08:08:28.199483 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.199459    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"]
Sep 14 08:08:28.213890 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.213862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86ebb312-4a36-4534-b09c-73ae4391ca27-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf\" (UID: \"86ebb312-4a36-4534-b09c-73ae4391ca27\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"
Sep 14 08:08:28.214019 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.213931    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8glj\" (UniqueName: \"kubernetes.io/projected/31cd80ad-a6a6-4cda-8368-4c06e8e25532-kube-api-access-x8glj\") pod \"payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b\" (UID: \"31cd80ad-a6a6-4cda-8368-4c06e8e25532\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"
Sep 14 08:08:28.214019 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.213991    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcn4k\" (UniqueName: \"kubernetes.io/projected/86ebb312-4a36-4534-b09c-73ae4391ca27-kube-api-access-lcn4k\") pod \"payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf\" (UID: \"86ebb312-4a36-4534-b09c-73ae4391ca27\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"
Sep 14 08:08:28.214134 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.214047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31cd80ad-a6a6-4cda-8368-4c06e8e25532-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b\" (UID: \"31cd80ad-a6a6-4cda-8368-4c06e8e25532\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"
Sep 14 08:08:28.315181 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.315146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcn4k\" (UniqueName: \"kubernetes.io/projected/86ebb312-4a36-4534-b09c-73ae4391ca27-kube-api-access-lcn4k\") pod \"payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf\" (UID: \"86ebb312-4a36-4534-b09c-73ae4391ca27\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"
Sep 14 08:08:28.315350 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.315190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31cd80ad-a6a6-4cda-8368-4c06e8e25532-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b\" (UID: \"31cd80ad-a6a6-4cda-8368-4c06e8e25532\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"
Sep 14 08:08:28.315416 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.315380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86ebb312-4a36-4534-b09c-73ae4391ca27-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf\" (UID: \"86ebb312-4a36-4534-b09c-73ae4391ca27\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"
Sep 14 08:08:28.315476 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.315454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8glj\" (UniqueName: \"kubernetes.io/projected/31cd80ad-a6a6-4cda-8368-4c06e8e25532-kube-api-access-x8glj\") pod \"payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b\" (UID: \"31cd80ad-a6a6-4cda-8368-4c06e8e25532\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"
Sep 14 08:08:28.315928 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.315909    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31cd80ad-a6a6-4cda-8368-4c06e8e25532-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b\" (UID: \"31cd80ad-a6a6-4cda-8368-4c06e8e25532\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"
Sep 14 08:08:28.316110 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.316078    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86ebb312-4a36-4534-b09c-73ae4391ca27-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf\" (UID: \"86ebb312-4a36-4534-b09c-73ae4391ca27\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"
Sep 14 08:08:28.325208 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.325186    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcn4k\" (UniqueName: \"kubernetes.io/projected/86ebb312-4a36-4534-b09c-73ae4391ca27-kube-api-access-lcn4k\") pod \"payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf\" (UID: \"86ebb312-4a36-4534-b09c-73ae4391ca27\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"
Sep 14 08:08:28.325333 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.325307    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8glj\" (UniqueName: \"kubernetes.io/projected/31cd80ad-a6a6-4cda-8368-4c06e8e25532-kube-api-access-x8glj\") pod \"payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b\" (UID: \"31cd80ad-a6a6-4cda-8368-4c06e8e25532\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"
Sep 14 08:08:28.463608 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.463517    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"
Sep 14 08:08:28.499855 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.499823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"
Sep 14 08:08:28.742820 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.742791    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"]
Sep 14 08:08:28.744537 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:08:28.744511    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31cd80ad_a6a6_4cda_8368_4c06e8e25532.slice/crio-ff20fb16b07b9928c55a7ff00665d93defc693500986a016334bbc4d97b0a100 WatchSource:0}: Error finding container ff20fb16b07b9928c55a7ff00665d93defc693500986a016334bbc4d97b0a100: Status 404 returned error can't find the container with id ff20fb16b07b9928c55a7ff00665d93defc693500986a016334bbc4d97b0a100
Sep 14 08:08:28.788861 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:28.788835    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"]
Sep 14 08:08:28.790396 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:08:28.790372    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86ebb312_4a36_4534_b09c_73ae4391ca27.slice/crio-62e322d883bf098a6209a8be0d05aa86567e8750d377e7f60f69d2fb3222b7aa WatchSource:0}: Error finding container 62e322d883bf098a6209a8be0d05aa86567e8750d377e7f60f69d2fb3222b7aa: Status 404 returned error can't find the container with id 62e322d883bf098a6209a8be0d05aa86567e8750d377e7f60f69d2fb3222b7aa
Sep 14 08:08:29.024664 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:29.024562    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b" event={"ID":"31cd80ad-a6a6-4cda-8368-4c06e8e25532","Type":"ContainerStarted","Data":"e221d2a87bd793999187c86841781277f1797c5c8702b84fcfc9c162e95ffa50"}
Sep 14 08:08:29.024839 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:29.024674    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b" event={"ID":"31cd80ad-a6a6-4cda-8368-4c06e8e25532","Type":"ContainerStarted","Data":"ff20fb16b07b9928c55a7ff00665d93defc693500986a016334bbc4d97b0a100"}
Sep 14 08:08:29.024839 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:29.024813    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"
Sep 14 08:08:29.026264 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:29.026236    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf" event={"ID":"86ebb312-4a36-4534-b09c-73ae4391ca27","Type":"ContainerStarted","Data":"64f75e4cfdd236f1f682a17ef4a11e25aa2e9d6e81eadbadbddf107da64cbfb6"}
Sep 14 08:08:29.026392 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:29.026278    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf" event={"ID":"86ebb312-4a36-4534-b09c-73ae4391ca27","Type":"ContainerStarted","Data":"62e322d883bf098a6209a8be0d05aa86567e8750d377e7f60f69d2fb3222b7aa"}
Sep 14 08:08:29.026392 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:29.026314    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"
Sep 14 08:08:29.042242 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:29.041783    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b" podStartSLOduration=1.041765977 podStartE2EDuration="1.041765977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:29.040621519 +0000 UTC m=+1229.669509148" watchObservedRunningTime="2026-09-14 08:08:29.041765977 +0000 UTC m=+1229.670653604"
Sep 14 08:08:29.059650 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:29.059606    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf" podStartSLOduration=1.05959172 podStartE2EDuration="1.05959172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:29.057825522 +0000 UTC m=+1229.686713172" watchObservedRunningTime="2026-09-14 08:08:29.05959172 +0000 UTC m=+1229.688479347"
Sep 14 08:08:40.035796 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:40.035764    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"
Sep 14 08:08:40.036267 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:08:40.035839    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"
Sep 14 08:09:35.325618 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.325578    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"]
Sep 14 08:09:35.326150 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.325914    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs" podUID="585eb966-1593-47d7-af00-e8cdb909a731" containerName="payload-processing" containerID="cri-o://7a75bac4444c9cbcfa0aada312d23390b616166cb45f86f0df3d7e0def5b3fab" gracePeriod=30
Sep 14 08:09:35.332904 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.332865    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"]
Sep 14 08:09:35.333473 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.333188    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl" podUID="de595a89-111e-410c-a56c-a65a5144cce8" containerName="payload-pre-processing" containerID="cri-o://90a731e62118371e8ffa7b26d4990d79b6795703a2fdea03aeea6022cf4ded55" gracePeriod=30
Sep 14 08:09:35.459795 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.459763    2563 generic.go:356] "Generic (PLEG): container finished" podID="de595a89-111e-410c-a56c-a65a5144cce8" containerID="90a731e62118371e8ffa7b26d4990d79b6795703a2fdea03aeea6022cf4ded55" exitCode=0
Sep 14 08:09:35.459964 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.459834    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl" event={"ID":"de595a89-111e-410c-a56c-a65a5144cce8","Type":"ContainerDied","Data":"90a731e62118371e8ffa7b26d4990d79b6795703a2fdea03aeea6022cf4ded55"}
Sep 14 08:09:35.461635 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.461609    2563 generic.go:356] "Generic (PLEG): container finished" podID="585eb966-1593-47d7-af00-e8cdb909a731" containerID="7a75bac4444c9cbcfa0aada312d23390b616166cb45f86f0df3d7e0def5b3fab" exitCode=0
Sep 14 08:09:35.461775 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.461684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs" event={"ID":"585eb966-1593-47d7-af00-e8cdb909a731","Type":"ContainerDied","Data":"7a75bac4444c9cbcfa0aada312d23390b616166cb45f86f0df3d7e0def5b3fab"}
Sep 14 08:09:35.617129 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.617102    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"
Sep 14 08:09:35.646702 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.643526    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"
Sep 14 08:09:35.772826 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.772796    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jp6w8\" (UniqueName: \"kubernetes.io/projected/de595a89-111e-410c-a56c-a65a5144cce8-kube-api-access-jp6w8\") pod \"de595a89-111e-410c-a56c-a65a5144cce8\" (UID: \"de595a89-111e-410c-a56c-a65a5144cce8\") "
Sep 14 08:09:35.772995 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.772840    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh7r9\" (UniqueName: \"kubernetes.io/projected/585eb966-1593-47d7-af00-e8cdb909a731-kube-api-access-rh7r9\") pod \"585eb966-1593-47d7-af00-e8cdb909a731\" (UID: \"585eb966-1593-47d7-af00-e8cdb909a731\") "
Sep 14 08:09:35.772995 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.772874    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/585eb966-1593-47d7-af00-e8cdb909a731-plugins-config-volume\") pod \"585eb966-1593-47d7-af00-e8cdb909a731\" (UID: \"585eb966-1593-47d7-af00-e8cdb909a731\") "
Sep 14 08:09:35.772995 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.772901    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de595a89-111e-410c-a56c-a65a5144cce8-plugins-config-volume\") pod \"de595a89-111e-410c-a56c-a65a5144cce8\" (UID: \"de595a89-111e-410c-a56c-a65a5144cce8\") "
Sep 14 08:09:35.773278 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.773245    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/585eb966-1593-47d7-af00-e8cdb909a731-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "585eb966-1593-47d7-af00-e8cdb909a731" (UID: "585eb966-1593-47d7-af00-e8cdb909a731"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:09:35.773363 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.773314    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de595a89-111e-410c-a56c-a65a5144cce8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "de595a89-111e-410c-a56c-a65a5144cce8" (UID: "de595a89-111e-410c-a56c-a65a5144cce8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:09:35.774864 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.774845    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/585eb966-1593-47d7-af00-e8cdb909a731-kube-api-access-rh7r9" (OuterVolumeSpecName: "kube-api-access-rh7r9") pod "585eb966-1593-47d7-af00-e8cdb909a731" (UID: "585eb966-1593-47d7-af00-e8cdb909a731"). InnerVolumeSpecName "kube-api-access-rh7r9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:35.774975 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.774933    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de595a89-111e-410c-a56c-a65a5144cce8-kube-api-access-jp6w8" (OuterVolumeSpecName: "kube-api-access-jp6w8") pod "de595a89-111e-410c-a56c-a65a5144cce8" (UID: "de595a89-111e-410c-a56c-a65a5144cce8"). InnerVolumeSpecName "kube-api-access-jp6w8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:35.874087 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.874005    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jp6w8\" (UniqueName: \"kubernetes.io/projected/de595a89-111e-410c-a56c-a65a5144cce8-kube-api-access-jp6w8\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:09:35.874087 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.874033    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh7r9\" (UniqueName: \"kubernetes.io/projected/585eb966-1593-47d7-af00-e8cdb909a731-kube-api-access-rh7r9\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:09:35.874087 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.874042    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/585eb966-1593-47d7-af00-e8cdb909a731-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:09:35.874087 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:35.874050    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de595a89-111e-410c-a56c-a65a5144cce8-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:09:36.469445 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:36.469413    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"
Sep 14 08:09:36.469861 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:36.469407    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl" event={"ID":"de595a89-111e-410c-a56c-a65a5144cce8","Type":"ContainerDied","Data":"54d409b3e0a3a2bbf20a93a539dca9b57476de8ea2c03272490723d2ffdac8f2"}
Sep 14 08:09:36.469861 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:36.469553    2563 scope.go:117] "RemoveContainer" containerID="90a731e62118371e8ffa7b26d4990d79b6795703a2fdea03aeea6022cf4ded55"
Sep 14 08:09:36.470972 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:36.470953    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"
Sep 14 08:09:36.471054 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:36.471009    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs" event={"ID":"585eb966-1593-47d7-af00-e8cdb909a731","Type":"ContainerDied","Data":"2c4888e39bdb003d71776fc9c6c7bfaf84fdb50c1b164950a9a4d6f9333d48ba"}
Sep 14 08:09:36.479803 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:36.479786    2563 scope.go:117] "RemoveContainer" containerID="7a75bac4444c9cbcfa0aada312d23390b616166cb45f86f0df3d7e0def5b3fab"
Sep 14 08:09:36.487796 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:36.487771    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"]
Sep 14 08:09:36.491548 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:36.491530    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b87debd-74df54cf78-tbntl"]
Sep 14 08:09:36.500069 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:36.500050    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"]
Sep 14 08:09:36.505628 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:36.505608    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8b87debd-68d5c96d7c-wwqfs"]
Sep 14 08:09:37.887022 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:37.886988    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="585eb966-1593-47d7-af00-e8cdb909a731" path="/var/lib/kubelet/pods/585eb966-1593-47d7-af00-e8cdb909a731/volumes"
Sep 14 08:09:37.887401 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:37.887368    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de595a89-111e-410c-a56c-a65a5144cce8" path="/var/lib/kubelet/pods/de595a89-111e-410c-a56c-a65a5144cce8/volumes"
Sep 14 08:09:54.705374 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.705334    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"]
Sep 14 08:09:54.706072 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.706049    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="585eb966-1593-47d7-af00-e8cdb909a731" containerName="payload-processing"
Sep 14 08:09:54.706072 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.706070    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="585eb966-1593-47d7-af00-e8cdb909a731" containerName="payload-processing"
Sep 14 08:09:54.706220 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.706087    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de595a89-111e-410c-a56c-a65a5144cce8" containerName="payload-pre-processing"
Sep 14 08:09:54.706220 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.706093    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="de595a89-111e-410c-a56c-a65a5144cce8" containerName="payload-pre-processing"
Sep 14 08:09:54.706220 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.706162    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="de595a89-111e-410c-a56c-a65a5144cce8" containerName="payload-pre-processing"
Sep 14 08:09:54.706220 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.706171    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="585eb966-1593-47d7-af00-e8cdb909a731" containerName="payload-processing"
Sep 14 08:09:54.711155 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.711134    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.714308 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.714187    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-ea9d35\"/\"kube-root-ca.crt\""
Sep 14 08:09:54.714308 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.714300    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-ea9d35\"/\"auth-test-model-ea9d35-kserve-self-signed-certs\""
Sep 14 08:09:54.714465 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.714429    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-ea9d35\"/\"openshift-service-ca.crt\""
Sep 14 08:09:54.714935 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.714914    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-ea9d35\"/\"default-dockercfg-5lrdl\""
Sep 14 08:09:54.717978 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.717953    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"]
Sep 14 08:09:54.846983 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.846944    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-home\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.847176 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.846990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6tkx\" (UniqueName: \"kubernetes.io/projected/f05750e9-c618-4aba-b38f-995682723fd9-kube-api-access-x6tkx\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.847176 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.847052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-model-cache\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.847176 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.847106    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-tmp-dir\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.847176 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.847133    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-dshm\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.847397 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.847258    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f05750e9-c618-4aba-b38f-995682723fd9-tls-certs\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.947889 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.947855    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f05750e9-c618-4aba-b38f-995682723fd9-tls-certs\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.948065 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.947894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-home\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.948065 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.947918    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6tkx\" (UniqueName: \"kubernetes.io/projected/f05750e9-c618-4aba-b38f-995682723fd9-kube-api-access-x6tkx\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.948065 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.947943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-model-cache\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.948238 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.948163    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-tmp-dir\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.948292 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.948244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-dshm\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.948353 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.948298    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-home\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.948411 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.948351    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-model-cache\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.948469 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.948447    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-tmp-dir\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.950460 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.950436    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-dshm\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.950571 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.950514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f05750e9-c618-4aba-b38f-995682723fd9-tls-certs\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:54.956697 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:54.956624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6tkx\" (UniqueName: \"kubernetes.io/projected/f05750e9-c618-4aba-b38f-995682723fd9-kube-api-access-x6tkx\") pod \"auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:55.022022 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:55.021996    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:09:55.303694 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:55.303660    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"]
Sep 14 08:09:55.305100 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:09:55.305070    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf05750e9_c618_4aba_b38f_995682723fd9.slice/crio-d342efa6a2793e513fb8f285810f7fefd633d9d5a588ff495abd968bf4030981 WatchSource:0}: Error finding container d342efa6a2793e513fb8f285810f7fefd633d9d5a588ff495abd968bf4030981: Status 404 returned error can't find the container with id d342efa6a2793e513fb8f285810f7fefd633d9d5a588ff495abd968bf4030981
Sep 14 08:09:55.589688 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:55.589632    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9" event={"ID":"f05750e9-c618-4aba-b38f-995682723fd9","Type":"ContainerStarted","Data":"d50d5508615854849d0350f4fdfd217d1fb1a5fbefb3e4798448425f532d3a9f"}
Sep 14 08:09:55.589848 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:55.589697    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9" event={"ID":"f05750e9-c618-4aba-b38f-995682723fd9","Type":"ContainerStarted","Data":"d342efa6a2793e513fb8f285810f7fefd633d9d5a588ff495abd968bf4030981"}
Sep 14 08:09:55.609478 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:09:55.609434    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9" podStartSLOduration=1.609419311 podStartE2EDuration="1.609419311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:55.605965002 +0000 UTC m=+1316.234852653" watchObservedRunningTime="2026-09-14 08:09:55.609419311 +0000 UTC m=+1316.238306934"
Sep 14 08:10:00.052824 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:10:00.052793    2563 scope.go:117] "RemoveContainer" containerID="081753d80042a1869e7175b1add261503ba6db0c8a510c2ce5f176af71a88c79"
Sep 14 08:10:05.022744 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:10:05.022710    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:10:05.023130 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:10:05.022815    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:10:05.035536 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:10:05.035511    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:10:05.662417 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:10:05.662390    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:11:29.068816 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.068778    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"]
Sep 14 08:11:29.072791 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.072768    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"
Sep 14 08:11:29.074634 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.074612    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-b914f7\""
Sep 14 08:11:29.074779 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.074699    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-b914f7-dockercfg-x2p27\""
Sep 14 08:11:29.082463 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.082436    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"]
Sep 14 08:11:29.102157 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.102128    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"]
Sep 14 08:11:29.106005 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.105984    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"
Sep 14 08:11:29.117904 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.117879    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"]
Sep 14 08:11:29.125580 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.125547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4h2t\" (UniqueName: \"kubernetes.io/projected/4688740a-752c-4185-8ee0-28a16b105294-kube-api-access-m4h2t\") pod \"payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn\" (UID: \"4688740a-752c-4185-8ee0-28a16b105294\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"
Sep 14 08:11:29.125725 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.125690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4688740a-752c-4185-8ee0-28a16b105294-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn\" (UID: \"4688740a-752c-4185-8ee0-28a16b105294\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"
Sep 14 08:11:29.226563 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.226529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4688740a-752c-4185-8ee0-28a16b105294-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn\" (UID: \"4688740a-752c-4185-8ee0-28a16b105294\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"
Sep 14 08:11:29.226734 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.226574    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77e81330-0faa-496f-a4bd-32db29f089fb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw\" (UID: \"77e81330-0faa-496f-a4bd-32db29f089fb\") " pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"
Sep 14 08:11:29.226734 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.226669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4h2t\" (UniqueName: \"kubernetes.io/projected/4688740a-752c-4185-8ee0-28a16b105294-kube-api-access-m4h2t\") pod \"payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn\" (UID: \"4688740a-752c-4185-8ee0-28a16b105294\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"
Sep 14 08:11:29.226734 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.226699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s486c\" (UniqueName: \"kubernetes.io/projected/77e81330-0faa-496f-a4bd-32db29f089fb-kube-api-access-s486c\") pod \"payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw\" (UID: \"77e81330-0faa-496f-a4bd-32db29f089fb\") " pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"
Sep 14 08:11:29.227336 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.227304    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4688740a-752c-4185-8ee0-28a16b105294-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn\" (UID: \"4688740a-752c-4185-8ee0-28a16b105294\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"
Sep 14 08:11:29.234139 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.234108    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4h2t\" (UniqueName: \"kubernetes.io/projected/4688740a-752c-4185-8ee0-28a16b105294-kube-api-access-m4h2t\") pod \"payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn\" (UID: \"4688740a-752c-4185-8ee0-28a16b105294\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"
Sep 14 08:11:29.328018 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.327976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s486c\" (UniqueName: \"kubernetes.io/projected/77e81330-0faa-496f-a4bd-32db29f089fb-kube-api-access-s486c\") pod \"payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw\" (UID: \"77e81330-0faa-496f-a4bd-32db29f089fb\") " pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"
Sep 14 08:11:29.328222 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.328083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77e81330-0faa-496f-a4bd-32db29f089fb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw\" (UID: \"77e81330-0faa-496f-a4bd-32db29f089fb\") " pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"
Sep 14 08:11:29.328810 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.328791    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77e81330-0faa-496f-a4bd-32db29f089fb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw\" (UID: \"77e81330-0faa-496f-a4bd-32db29f089fb\") " pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"
Sep 14 08:11:29.336028 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.335999    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s486c\" (UniqueName: \"kubernetes.io/projected/77e81330-0faa-496f-a4bd-32db29f089fb-kube-api-access-s486c\") pod \"payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw\" (UID: \"77e81330-0faa-496f-a4bd-32db29f089fb\") " pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"
Sep 14 08:11:29.385415 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.385379    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"
Sep 14 08:11:29.419035 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.419004    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"
Sep 14 08:11:29.661788 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.661753    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"]
Sep 14 08:11:29.662815 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:11:29.662783    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4688740a_752c_4185_8ee0_28a16b105294.slice/crio-5b58480addbb8be44b1c823f129c7b7310b7684dcafdc1ef335ba43ef007b8f2 WatchSource:0}: Error finding container 5b58480addbb8be44b1c823f129c7b7310b7684dcafdc1ef335ba43ef007b8f2: Status 404 returned error can't find the container with id 5b58480addbb8be44b1c823f129c7b7310b7684dcafdc1ef335ba43ef007b8f2
Sep 14 08:11:29.718706 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:11:29.718681    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77e81330_0faa_496f_a4bd_32db29f089fb.slice/crio-6ac8e2acf150f890ce70820f97108b2048f48ab7dd9c0922e2bcff68d532f72a WatchSource:0}: Error finding container 6ac8e2acf150f890ce70820f97108b2048f48ab7dd9c0922e2bcff68d532f72a: Status 404 returned error can't find the container with id 6ac8e2acf150f890ce70820f97108b2048f48ab7dd9c0922e2bcff68d532f72a
Sep 14 08:11:29.718886 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:29.718865    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"]
Sep 14 08:11:30.179028 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:30.178993    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw" event={"ID":"77e81330-0faa-496f-a4bd-32db29f089fb","Type":"ContainerStarted","Data":"c7a2029d6716d6fc1da85e0bb5f9d6b6f126d30282464fb15b9b147d35c4fc3e"}
Sep 14 08:11:30.179477 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:30.179065    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw" event={"ID":"77e81330-0faa-496f-a4bd-32db29f089fb","Type":"ContainerStarted","Data":"6ac8e2acf150f890ce70820f97108b2048f48ab7dd9c0922e2bcff68d532f72a"}
Sep 14 08:11:30.179477 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:30.179156    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"
Sep 14 08:11:30.180852 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:30.180817    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn" event={"ID":"4688740a-752c-4185-8ee0-28a16b105294","Type":"ContainerStarted","Data":"5dadc38c4fdea4fced4a289ee704b6ec20bfb1a5608197079c98587881a689bc"}
Sep 14 08:11:30.181065 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:30.180877    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn" event={"ID":"4688740a-752c-4185-8ee0-28a16b105294","Type":"ContainerStarted","Data":"5b58480addbb8be44b1c823f129c7b7310b7684dcafdc1ef335ba43ef007b8f2"}
Sep 14 08:11:30.181065 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:30.180903    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"
Sep 14 08:11:30.195947 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:30.195902    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw" podStartSLOduration=1.195887986 podStartE2EDuration="1.195887986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:11:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:11:30.193822963 +0000 UTC m=+1410.822710613" watchObservedRunningTime="2026-09-14 08:11:30.195887986 +0000 UTC m=+1410.824775613"
Sep 14 08:11:30.211154 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:30.211107    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn" podStartSLOduration=1.211092924 podStartE2EDuration="1.211092924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:11:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:11:30.20841896 +0000 UTC m=+1410.837306586" watchObservedRunningTime="2026-09-14 08:11:30.211092924 +0000 UTC m=+1410.839980550"
Sep 14 08:11:41.190281 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:41.190251    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"
Sep 14 08:11:41.190657 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:11:41.190323    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"
Sep 14 08:12:03.574269 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:03.574179    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"]
Sep 14 08:12:03.578541 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:03.578515    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"
Sep 14 08:12:03.580940 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:03.580916    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-87b7cb\""
Sep 14 08:12:03.581072 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:03.581034    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-87b7cb-dockercfg-vhfv6\""
Sep 14 08:12:03.589620 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:03.589594    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"]
Sep 14 08:12:03.663111 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:03.663070    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9c75\" (UniqueName: \"kubernetes.io/projected/edc3aa79-bb42-4a98-b462-932104c07280-kube-api-access-j9c75\") pod \"payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg\" (UID: \"edc3aa79-bb42-4a98-b462-932104c07280\") " pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"
Sep 14 08:12:03.663274 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:03.663156    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edc3aa79-bb42-4a98-b462-932104c07280-plugins-config-volume\") pod \"payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg\" (UID: \"edc3aa79-bb42-4a98-b462-932104c07280\") " pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"
Sep 14 08:12:03.764268 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:03.764227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9c75\" (UniqueName: \"kubernetes.io/projected/edc3aa79-bb42-4a98-b462-932104c07280-kube-api-access-j9c75\") pod \"payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg\" (UID: \"edc3aa79-bb42-4a98-b462-932104c07280\") " pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"
Sep 14 08:12:03.764450 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:03.764311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edc3aa79-bb42-4a98-b462-932104c07280-plugins-config-volume\") pod \"payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg\" (UID: \"edc3aa79-bb42-4a98-b462-932104c07280\") " pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"
Sep 14 08:12:03.765023 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:03.765004    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edc3aa79-bb42-4a98-b462-932104c07280-plugins-config-volume\") pod \"payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg\" (UID: \"edc3aa79-bb42-4a98-b462-932104c07280\") " pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"
Sep 14 08:12:03.773458 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:03.773430    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9c75\" (UniqueName: \"kubernetes.io/projected/edc3aa79-bb42-4a98-b462-932104c07280-kube-api-access-j9c75\") pod \"payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg\" (UID: \"edc3aa79-bb42-4a98-b462-932104c07280\") " pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"
Sep 14 08:12:03.889923 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:03.889887    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"
Sep 14 08:12:04.168129 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:04.168078    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"]
Sep 14 08:12:04.168583 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:12:04.168549    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedc3aa79_bb42_4a98_b462_932104c07280.slice/crio-5b44ab939dacc440c5083e1cceb9c0e6cb18b114a692c39ffe187d46aa134a3b WatchSource:0}: Error finding container 5b44ab939dacc440c5083e1cceb9c0e6cb18b114a692c39ffe187d46aa134a3b: Status 404 returned error can't find the container with id 5b44ab939dacc440c5083e1cceb9c0e6cb18b114a692c39ffe187d46aa134a3b
Sep 14 08:12:04.398771 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:04.398718    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg" event={"ID":"edc3aa79-bb42-4a98-b462-932104c07280","Type":"ContainerStarted","Data":"f22c7fd4d877281d2e37cb793964c8874edebaef30cba2efd9fb7d9a81098f19"}
Sep 14 08:12:04.398981 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:04.398789    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"
Sep 14 08:12:04.398981 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:04.398809    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg" event={"ID":"edc3aa79-bb42-4a98-b462-932104c07280","Type":"ContainerStarted","Data":"5b44ab939dacc440c5083e1cceb9c0e6cb18b114a692c39ffe187d46aa134a3b"}
Sep 14 08:12:04.422614 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:04.422493    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg" podStartSLOduration=1.4224781069999999 podStartE2EDuration="1.422478107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:12:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:12:04.419633025 +0000 UTC m=+1445.048520687" watchObservedRunningTime="2026-09-14 08:12:04.422478107 +0000 UTC m=+1445.051365733"
Sep 14 08:12:09.228136 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:09.228099    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v"]
Sep 14 08:12:09.231429 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:09.231403    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v"
Sep 14 08:12:09.234842 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:09.234815    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:12:09.247990 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:09.247962    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v"]
Sep 14 08:12:09.313015 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:09.312985    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6876e3dd-b692-46ac-9244-16a92cee4e17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v\" (UID: \"6876e3dd-b692-46ac-9244-16a92cee4e17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v"
Sep 14 08:12:09.413863 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:09.413833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6876e3dd-b692-46ac-9244-16a92cee4e17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v\" (UID: \"6876e3dd-b692-46ac-9244-16a92cee4e17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v"
Sep 14 08:12:09.414431 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:09.414413    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6876e3dd-b692-46ac-9244-16a92cee4e17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v\" (UID: \"6876e3dd-b692-46ac-9244-16a92cee4e17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v"
Sep 14 08:12:09.551997 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:09.551914    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v"
Sep 14 08:12:09.828636 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:09.828609    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v"]
Sep 14 08:12:09.829399 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:12:09.829365    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6876e3dd_b692_46ac_9244_16a92cee4e17.slice/crio-b8e243055c67b57c23711dfefa2cb2467f29549148e8a39e98852f84d74414d5 WatchSource:0}: Error finding container b8e243055c67b57c23711dfefa2cb2467f29549148e8a39e98852f84d74414d5: Status 404 returned error can't find the container with id b8e243055c67b57c23711dfefa2cb2467f29549148e8a39e98852f84d74414d5
Sep 14 08:12:10.437139 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:10.437037    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v" event={"ID":"6876e3dd-b692-46ac-9244-16a92cee4e17","Type":"ContainerStarted","Data":"d0d5669951315f18a8f624802b96e42f7c3f067fc9a505f1b76d147c708e6c36"}
Sep 14 08:12:10.437139 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:10.437094    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v" event={"ID":"6876e3dd-b692-46ac-9244-16a92cee4e17","Type":"ContainerStarted","Data":"b8e243055c67b57c23711dfefa2cb2467f29549148e8a39e98852f84d74414d5"}
Sep 14 08:12:10.453220 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:10.453169    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v" podStartSLOduration=1.453153174 podStartE2EDuration="1.453153174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:12:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:12:10.452279252 +0000 UTC m=+1451.081166906" watchObservedRunningTime="2026-09-14 08:12:10.453153174 +0000 UTC m=+1451.082040851"
Sep 14 08:12:11.445790 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:11.445759    2563 generic.go:356] "Generic (PLEG): container finished" podID="6876e3dd-b692-46ac-9244-16a92cee4e17" containerID="d0d5669951315f18a8f624802b96e42f7c3f067fc9a505f1b76d147c708e6c36" exitCode=0
Sep 14 08:12:11.446235 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:11.445853    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v" event={"ID":"6876e3dd-b692-46ac-9244-16a92cee4e17","Type":"ContainerDied","Data":"d0d5669951315f18a8f624802b96e42f7c3f067fc9a505f1b76d147c708e6c36"}
Sep 14 08:12:12.588824 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:12.588800    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v"
Sep 14 08:12:12.646918 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:12.646887    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6876e3dd-b692-46ac-9244-16a92cee4e17-maas-api-service-ca\") pod \"6876e3dd-b692-46ac-9244-16a92cee4e17\" (UID: \"6876e3dd-b692-46ac-9244-16a92cee4e17\") "
Sep 14 08:12:12.647230 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:12.647208    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6876e3dd-b692-46ac-9244-16a92cee4e17-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6876e3dd-b692-46ac-9244-16a92cee4e17" (UID: "6876e3dd-b692-46ac-9244-16a92cee4e17"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:12.647316 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:12.647297    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6876e3dd-b692-46ac-9244-16a92cee4e17-maas-api-service-ca\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:12:13.459271 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:13.459232    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v" event={"ID":"6876e3dd-b692-46ac-9244-16a92cee4e17","Type":"ContainerDied","Data":"b8e243055c67b57c23711dfefa2cb2467f29549148e8a39e98852f84d74414d5"}
Sep 14 08:12:13.459480 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:13.459288    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8e243055c67b57c23711dfefa2cb2467f29549148e8a39e98852f84d74414d5"
Sep 14 08:12:13.459480 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:13.459242    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v"
Sep 14 08:12:15.407471 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:15.407438    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"
Sep 14 08:12:20.837216 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:20.837176    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv"]
Sep 14 08:12:20.837926 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:20.837906    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6876e3dd-b692-46ac-9244-16a92cee4e17" containerName="revoke-keys"
Sep 14 08:12:20.838030 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:20.837929    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6876e3dd-b692-46ac-9244-16a92cee4e17" containerName="revoke-keys"
Sep 14 08:12:20.838085 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:20.838045    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6876e3dd-b692-46ac-9244-16a92cee4e17" containerName="revoke-keys"
Sep 14 08:12:20.841603 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:20.841577    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv"
Sep 14 08:12:20.845075 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:20.845049    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:12:20.850666 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:20.850627    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv"]
Sep 14 08:12:20.931198 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:20.931159    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314e4541-8302-47e9-92ad-3a5df13bcd36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b914f7-sclqv\" (UID: \"314e4541-8302-47e9-92ad-3a5df13bcd36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv"
Sep 14 08:12:21.032378 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:21.032343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314e4541-8302-47e9-92ad-3a5df13bcd36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b914f7-sclqv\" (UID: \"314e4541-8302-47e9-92ad-3a5df13bcd36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv"
Sep 14 08:12:21.033031 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:21.033012    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314e4541-8302-47e9-92ad-3a5df13bcd36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b914f7-sclqv\" (UID: \"314e4541-8302-47e9-92ad-3a5df13bcd36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv"
Sep 14 08:12:21.155312 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:21.155229    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv"
Sep 14 08:12:21.433606 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:21.433572    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv"]
Sep 14 08:12:21.434385 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:12:21.434345    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod314e4541_8302_47e9_92ad_3a5df13bcd36.slice/crio-c21e365c6f15aa8e88e66c9c49a6f4737138faf0b778568112dfc3ebdbc7f10b WatchSource:0}: Error finding container c21e365c6f15aa8e88e66c9c49a6f4737138faf0b778568112dfc3ebdbc7f10b: Status 404 returned error can't find the container with id c21e365c6f15aa8e88e66c9c49a6f4737138faf0b778568112dfc3ebdbc7f10b
Sep 14 08:12:21.511281 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:21.511251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv" event={"ID":"314e4541-8302-47e9-92ad-3a5df13bcd36","Type":"ContainerStarted","Data":"c21e365c6f15aa8e88e66c9c49a6f4737138faf0b778568112dfc3ebdbc7f10b"}
Sep 14 08:12:22.519794 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:22.519714    2563 generic.go:356] "Generic (PLEG): container finished" podID="314e4541-8302-47e9-92ad-3a5df13bcd36" containerID="62aef19086ddaee0552e884f638d22363720c0e731f00c16b8363599e8c2ba1f" exitCode=0
Sep 14 08:12:22.520172 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:22.519804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv" event={"ID":"314e4541-8302-47e9-92ad-3a5df13bcd36","Type":"ContainerDied","Data":"62aef19086ddaee0552e884f638d22363720c0e731f00c16b8363599e8c2ba1f"}
Sep 14 08:12:23.681525 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:23.681502    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv"
Sep 14 08:12:23.862000 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:23.861963    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314e4541-8302-47e9-92ad-3a5df13bcd36-maas-api-service-ca\") pod \"314e4541-8302-47e9-92ad-3a5df13bcd36\" (UID: \"314e4541-8302-47e9-92ad-3a5df13bcd36\") "
Sep 14 08:12:23.862379 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:23.862359    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/314e4541-8302-47e9-92ad-3a5df13bcd36-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "314e4541-8302-47e9-92ad-3a5df13bcd36" (UID: "314e4541-8302-47e9-92ad-3a5df13bcd36"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:23.963131 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:23.963095    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314e4541-8302-47e9-92ad-3a5df13bcd36-maas-api-service-ca\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:12:24.534181 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:24.534152    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv"
Sep 14 08:12:24.534348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:24.534178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv" event={"ID":"314e4541-8302-47e9-92ad-3a5df13bcd36","Type":"ContainerDied","Data":"c21e365c6f15aa8e88e66c9c49a6f4737138faf0b778568112dfc3ebdbc7f10b"}
Sep 14 08:12:24.534348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:24.534246    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c21e365c6f15aa8e88e66c9c49a6f4737138faf0b778568112dfc3ebdbc7f10b"
Sep 14 08:12:27.897514 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:27.897464    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"]
Sep 14 08:12:27.898268 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:27.898217    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj" podUID="ab0c3b13-6781-4162-ac9c-85416ac28bae" containerName="payload-processing" containerID="cri-o://57e06e369968ceb7a5ccd7750c4d874d0a7e6c95d6fa4b339a30de9dab3f6fc1" gracePeriod=30
Sep 14 08:12:27.902758 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:27.902732    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"]
Sep 14 08:12:27.902969 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:27.902945    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k" podUID="66336fc7-cbac-4183-afbf-d8e005cdcbbb" containerName="payload-pre-processing" containerID="cri-o://1eeeb28d551dce83e75680137740fc255ac61b1128fdbc01ecc80f7574920d30" gracePeriod=30
Sep 14 08:12:28.067426 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.067367    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v"]
Sep 14 08:12:28.070370 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.070338    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-b69a34-jrs9v"]
Sep 14 08:12:28.179409 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.179383    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"
Sep 14 08:12:28.210249 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.210223    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"
Sep 14 08:12:28.302251 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.302218    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab0c3b13-6781-4162-ac9c-85416ac28bae-plugins-config-volume\") pod \"ab0c3b13-6781-4162-ac9c-85416ac28bae\" (UID: \"ab0c3b13-6781-4162-ac9c-85416ac28bae\") "
Sep 14 08:12:28.302428 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.302397    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcgss\" (UniqueName: \"kubernetes.io/projected/ab0c3b13-6781-4162-ac9c-85416ac28bae-kube-api-access-rcgss\") pod \"ab0c3b13-6781-4162-ac9c-85416ac28bae\" (UID: \"ab0c3b13-6781-4162-ac9c-85416ac28bae\") "
Sep 14 08:12:28.302636 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.302613    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab0c3b13-6781-4162-ac9c-85416ac28bae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ab0c3b13-6781-4162-ac9c-85416ac28bae" (UID: "ab0c3b13-6781-4162-ac9c-85416ac28bae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:28.304361 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.304340    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab0c3b13-6781-4162-ac9c-85416ac28bae-kube-api-access-rcgss" (OuterVolumeSpecName: "kube-api-access-rcgss") pod "ab0c3b13-6781-4162-ac9c-85416ac28bae" (UID: "ab0c3b13-6781-4162-ac9c-85416ac28bae"). InnerVolumeSpecName "kube-api-access-rcgss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:28.403238 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.403206    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66336fc7-cbac-4183-afbf-d8e005cdcbbb-plugins-config-volume\") pod \"66336fc7-cbac-4183-afbf-d8e005cdcbbb\" (UID: \"66336fc7-cbac-4183-afbf-d8e005cdcbbb\") "
Sep 14 08:12:28.403405 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.403329    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4v7mp\" (UniqueName: \"kubernetes.io/projected/66336fc7-cbac-4183-afbf-d8e005cdcbbb-kube-api-access-4v7mp\") pod \"66336fc7-cbac-4183-afbf-d8e005cdcbbb\" (UID: \"66336fc7-cbac-4183-afbf-d8e005cdcbbb\") "
Sep 14 08:12:28.403528 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.403503    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66336fc7-cbac-4183-afbf-d8e005cdcbbb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "66336fc7-cbac-4183-afbf-d8e005cdcbbb" (UID: "66336fc7-cbac-4183-afbf-d8e005cdcbbb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:28.403607 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.403515    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcgss\" (UniqueName: \"kubernetes.io/projected/ab0c3b13-6781-4162-ac9c-85416ac28bae-kube-api-access-rcgss\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:12:28.403607 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.403555    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab0c3b13-6781-4162-ac9c-85416ac28bae-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:12:28.405390 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.405332    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66336fc7-cbac-4183-afbf-d8e005cdcbbb-kube-api-access-4v7mp" (OuterVolumeSpecName: "kube-api-access-4v7mp") pod "66336fc7-cbac-4183-afbf-d8e005cdcbbb" (UID: "66336fc7-cbac-4183-afbf-d8e005cdcbbb"). InnerVolumeSpecName "kube-api-access-4v7mp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:28.505037 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.505003    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66336fc7-cbac-4183-afbf-d8e005cdcbbb-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:12:28.505037 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.505032    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4v7mp\" (UniqueName: \"kubernetes.io/projected/66336fc7-cbac-4183-afbf-d8e005cdcbbb-kube-api-access-4v7mp\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:12:28.565277 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.565240    2563 generic.go:356] "Generic (PLEG): container finished" podID="66336fc7-cbac-4183-afbf-d8e005cdcbbb" containerID="1eeeb28d551dce83e75680137740fc255ac61b1128fdbc01ecc80f7574920d30" exitCode=0
Sep 14 08:12:28.565461 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.565282    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"
Sep 14 08:12:28.565461 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.565321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k" event={"ID":"66336fc7-cbac-4183-afbf-d8e005cdcbbb","Type":"ContainerDied","Data":"1eeeb28d551dce83e75680137740fc255ac61b1128fdbc01ecc80f7574920d30"}
Sep 14 08:12:28.565461 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.565390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k" event={"ID":"66336fc7-cbac-4183-afbf-d8e005cdcbbb","Type":"ContainerDied","Data":"538dd4050f3c7dc60223023ab9700e89936f9063b677cd77af71132a991d9acc"}
Sep 14 08:12:28.565461 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.565419    2563 scope.go:117] "RemoveContainer" containerID="1eeeb28d551dce83e75680137740fc255ac61b1128fdbc01ecc80f7574920d30"
Sep 14 08:12:28.575869 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.575725    2563 generic.go:356] "Generic (PLEG): container finished" podID="ab0c3b13-6781-4162-ac9c-85416ac28bae" containerID="57e06e369968ceb7a5ccd7750c4d874d0a7e6c95d6fa4b339a30de9dab3f6fc1" exitCode=0
Sep 14 08:12:28.575869 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.575763    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj" event={"ID":"ab0c3b13-6781-4162-ac9c-85416ac28bae","Type":"ContainerDied","Data":"57e06e369968ceb7a5ccd7750c4d874d0a7e6c95d6fa4b339a30de9dab3f6fc1"}
Sep 14 08:12:28.575869 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.575796    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"
Sep 14 08:12:28.575869 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.575857    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj" event={"ID":"ab0c3b13-6781-4162-ac9c-85416ac28bae","Type":"ContainerDied","Data":"7ca2e77a501978f53cbaf75d9227096f8010c932089750e297b38053048b1671"}
Sep 14 08:12:28.584945 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.584928    2563 scope.go:117] "RemoveContainer" containerID="1eeeb28d551dce83e75680137740fc255ac61b1128fdbc01ecc80f7574920d30"
Sep 14 08:12:28.585190 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:12:28.585172    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eeeb28d551dce83e75680137740fc255ac61b1128fdbc01ecc80f7574920d30\": container with ID starting with 1eeeb28d551dce83e75680137740fc255ac61b1128fdbc01ecc80f7574920d30 not found: ID does not exist" containerID="1eeeb28d551dce83e75680137740fc255ac61b1128fdbc01ecc80f7574920d30"
Sep 14 08:12:28.585240 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.585197    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eeeb28d551dce83e75680137740fc255ac61b1128fdbc01ecc80f7574920d30"} err="failed to get container status \"1eeeb28d551dce83e75680137740fc255ac61b1128fdbc01ecc80f7574920d30\": rpc error: code = NotFound desc = could not find container \"1eeeb28d551dce83e75680137740fc255ac61b1128fdbc01ecc80f7574920d30\": container with ID starting with 1eeeb28d551dce83e75680137740fc255ac61b1128fdbc01ecc80f7574920d30 not found: ID does not exist"
Sep 14 08:12:28.585240 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.585213    2563 scope.go:117] "RemoveContainer" containerID="57e06e369968ceb7a5ccd7750c4d874d0a7e6c95d6fa4b339a30de9dab3f6fc1"
Sep 14 08:12:28.593891 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.593870    2563 scope.go:117] "RemoveContainer" containerID="57e06e369968ceb7a5ccd7750c4d874d0a7e6c95d6fa4b339a30de9dab3f6fc1"
Sep 14 08:12:28.594175 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:12:28.594155    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57e06e369968ceb7a5ccd7750c4d874d0a7e6c95d6fa4b339a30de9dab3f6fc1\": container with ID starting with 57e06e369968ceb7a5ccd7750c4d874d0a7e6c95d6fa4b339a30de9dab3f6fc1 not found: ID does not exist" containerID="57e06e369968ceb7a5ccd7750c4d874d0a7e6c95d6fa4b339a30de9dab3f6fc1"
Sep 14 08:12:28.594226 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.594181    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57e06e369968ceb7a5ccd7750c4d874d0a7e6c95d6fa4b339a30de9dab3f6fc1"} err="failed to get container status \"57e06e369968ceb7a5ccd7750c4d874d0a7e6c95d6fa4b339a30de9dab3f6fc1\": rpc error: code = NotFound desc = could not find container \"57e06e369968ceb7a5ccd7750c4d874d0a7e6c95d6fa4b339a30de9dab3f6fc1\": container with ID starting with 57e06e369968ceb7a5ccd7750c4d874d0a7e6c95d6fa4b339a30de9dab3f6fc1 not found: ID does not exist"
Sep 14 08:12:28.594226 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.594188    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"]
Sep 14 08:12:28.599948 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.599926    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b69a34-5f69df5d66-t8b2k"]
Sep 14 08:12:28.609389 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.609364    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"]
Sep 14 08:12:28.613166 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:28.613147    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b69a34-5478797dd5-ctsgj"]
Sep 14 08:12:29.887996 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:29.887961    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66336fc7-cbac-4183-afbf-d8e005cdcbbb" path="/var/lib/kubelet/pods/66336fc7-cbac-4183-afbf-d8e005cdcbbb/volumes"
Sep 14 08:12:29.888414 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:29.888306    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6876e3dd-b692-46ac-9244-16a92cee4e17" path="/var/lib/kubelet/pods/6876e3dd-b692-46ac-9244-16a92cee4e17/volumes"
Sep 14 08:12:29.888626 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:29.888613    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab0c3b13-6781-4162-ac9c-85416ac28bae" path="/var/lib/kubelet/pods/ab0c3b13-6781-4162-ac9c-85416ac28bae/volumes"
Sep 14 08:12:38.927404 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:38.927367    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"]
Sep 14 08:12:38.927894 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:38.927610    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw" podUID="77e81330-0faa-496f-a4bd-32db29f089fb" containerName="payload-processing" containerID="cri-o://c7a2029d6716d6fc1da85e0bb5f9d6b6f126d30282464fb15b9b147d35c4fc3e" gracePeriod=30
Sep 14 08:12:38.937513 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:38.937483    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"]
Sep 14 08:12:38.937833 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:38.937804    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn" podUID="4688740a-752c-4185-8ee0-28a16b105294" containerName="payload-pre-processing" containerID="cri-o://5dadc38c4fdea4fced4a289ee704b6ec20bfb1a5608197079c98587881a689bc" gracePeriod=30
Sep 14 08:12:39.114924 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.114862    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv"]
Sep 14 08:12:39.121067 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.120964    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b914f7-sclqv"]
Sep 14 08:12:39.208348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.208326    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"
Sep 14 08:12:39.237500 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.237474    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"
Sep 14 08:12:39.298604 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.298576    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77e81330-0faa-496f-a4bd-32db29f089fb-plugins-config-volume\") pod \"77e81330-0faa-496f-a4bd-32db29f089fb\" (UID: \"77e81330-0faa-496f-a4bd-32db29f089fb\") "
Sep 14 08:12:39.298766 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.298687    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4688740a-752c-4185-8ee0-28a16b105294-plugins-config-volume\") pod \"4688740a-752c-4185-8ee0-28a16b105294\" (UID: \"4688740a-752c-4185-8ee0-28a16b105294\") "
Sep 14 08:12:39.298766 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.298723    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s486c\" (UniqueName: \"kubernetes.io/projected/77e81330-0faa-496f-a4bd-32db29f089fb-kube-api-access-s486c\") pod \"77e81330-0faa-496f-a4bd-32db29f089fb\" (UID: \"77e81330-0faa-496f-a4bd-32db29f089fb\") "
Sep 14 08:12:39.298842 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.298785    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4h2t\" (UniqueName: \"kubernetes.io/projected/4688740a-752c-4185-8ee0-28a16b105294-kube-api-access-m4h2t\") pod \"4688740a-752c-4185-8ee0-28a16b105294\" (UID: \"4688740a-752c-4185-8ee0-28a16b105294\") "
Sep 14 08:12:39.299028 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.299006    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77e81330-0faa-496f-a4bd-32db29f089fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "77e81330-0faa-496f-a4bd-32db29f089fb" (UID: "77e81330-0faa-496f-a4bd-32db29f089fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:39.299117 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.299108    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77e81330-0faa-496f-a4bd-32db29f089fb-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:12:39.299175 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.299095    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4688740a-752c-4185-8ee0-28a16b105294-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4688740a-752c-4185-8ee0-28a16b105294" (UID: "4688740a-752c-4185-8ee0-28a16b105294"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:39.301216 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.301175    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77e81330-0faa-496f-a4bd-32db29f089fb-kube-api-access-s486c" (OuterVolumeSpecName: "kube-api-access-s486c") pod "77e81330-0faa-496f-a4bd-32db29f089fb" (UID: "77e81330-0faa-496f-a4bd-32db29f089fb"). InnerVolumeSpecName "kube-api-access-s486c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:39.301321 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.301227    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4688740a-752c-4185-8ee0-28a16b105294-kube-api-access-m4h2t" (OuterVolumeSpecName: "kube-api-access-m4h2t") pod "4688740a-752c-4185-8ee0-28a16b105294" (UID: "4688740a-752c-4185-8ee0-28a16b105294"). InnerVolumeSpecName "kube-api-access-m4h2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:39.400107 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.400074    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4688740a-752c-4185-8ee0-28a16b105294-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:12:39.400107 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.400104    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s486c\" (UniqueName: \"kubernetes.io/projected/77e81330-0faa-496f-a4bd-32db29f089fb-kube-api-access-s486c\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:12:39.400107 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.400114    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4h2t\" (UniqueName: \"kubernetes.io/projected/4688740a-752c-4185-8ee0-28a16b105294-kube-api-access-m4h2t\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:12:39.647907 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.647871    2563 generic.go:356] "Generic (PLEG): container finished" podID="4688740a-752c-4185-8ee0-28a16b105294" containerID="5dadc38c4fdea4fced4a289ee704b6ec20bfb1a5608197079c98587881a689bc" exitCode=0
Sep 14 08:12:39.648070 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.647930    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"
Sep 14 08:12:39.648070 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.647951    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn" event={"ID":"4688740a-752c-4185-8ee0-28a16b105294","Type":"ContainerDied","Data":"5dadc38c4fdea4fced4a289ee704b6ec20bfb1a5608197079c98587881a689bc"}
Sep 14 08:12:39.648070 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.648048    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn" event={"ID":"4688740a-752c-4185-8ee0-28a16b105294","Type":"ContainerDied","Data":"5b58480addbb8be44b1c823f129c7b7310b7684dcafdc1ef335ba43ef007b8f2"}
Sep 14 08:12:39.648223 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.648076    2563 scope.go:117] "RemoveContainer" containerID="5dadc38c4fdea4fced4a289ee704b6ec20bfb1a5608197079c98587881a689bc"
Sep 14 08:12:39.649604 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.649580    2563 generic.go:356] "Generic (PLEG): container finished" podID="77e81330-0faa-496f-a4bd-32db29f089fb" containerID="c7a2029d6716d6fc1da85e0bb5f9d6b6f126d30282464fb15b9b147d35c4fc3e" exitCode=0
Sep 14 08:12:39.649771 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.649668    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw" event={"ID":"77e81330-0faa-496f-a4bd-32db29f089fb","Type":"ContainerDied","Data":"c7a2029d6716d6fc1da85e0bb5f9d6b6f126d30282464fb15b9b147d35c4fc3e"}
Sep 14 08:12:39.649771 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.649716    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw" event={"ID":"77e81330-0faa-496f-a4bd-32db29f089fb","Type":"ContainerDied","Data":"6ac8e2acf150f890ce70820f97108b2048f48ab7dd9c0922e2bcff68d532f72a"}
Sep 14 08:12:39.649771 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.649671    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"
Sep 14 08:12:39.659974 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.659909    2563 scope.go:117] "RemoveContainer" containerID="5dadc38c4fdea4fced4a289ee704b6ec20bfb1a5608197079c98587881a689bc"
Sep 14 08:12:39.660281 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:12:39.660235    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dadc38c4fdea4fced4a289ee704b6ec20bfb1a5608197079c98587881a689bc\": container with ID starting with 5dadc38c4fdea4fced4a289ee704b6ec20bfb1a5608197079c98587881a689bc not found: ID does not exist" containerID="5dadc38c4fdea4fced4a289ee704b6ec20bfb1a5608197079c98587881a689bc"
Sep 14 08:12:39.660281 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.660267    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dadc38c4fdea4fced4a289ee704b6ec20bfb1a5608197079c98587881a689bc"} err="failed to get container status \"5dadc38c4fdea4fced4a289ee704b6ec20bfb1a5608197079c98587881a689bc\": rpc error: code = NotFound desc = could not find container \"5dadc38c4fdea4fced4a289ee704b6ec20bfb1a5608197079c98587881a689bc\": container with ID starting with 5dadc38c4fdea4fced4a289ee704b6ec20bfb1a5608197079c98587881a689bc not found: ID does not exist"
Sep 14 08:12:39.660450 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.660293    2563 scope.go:117] "RemoveContainer" containerID="c7a2029d6716d6fc1da85e0bb5f9d6b6f126d30282464fb15b9b147d35c4fc3e"
Sep 14 08:12:39.668896 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.668876    2563 scope.go:117] "RemoveContainer" containerID="c7a2029d6716d6fc1da85e0bb5f9d6b6f126d30282464fb15b9b147d35c4fc3e"
Sep 14 08:12:39.669145 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:12:39.669122    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7a2029d6716d6fc1da85e0bb5f9d6b6f126d30282464fb15b9b147d35c4fc3e\": container with ID starting with c7a2029d6716d6fc1da85e0bb5f9d6b6f126d30282464fb15b9b147d35c4fc3e not found: ID does not exist" containerID="c7a2029d6716d6fc1da85e0bb5f9d6b6f126d30282464fb15b9b147d35c4fc3e"
Sep 14 08:12:39.669194 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.669155    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7a2029d6716d6fc1da85e0bb5f9d6b6f126d30282464fb15b9b147d35c4fc3e"} err="failed to get container status \"c7a2029d6716d6fc1da85e0bb5f9d6b6f126d30282464fb15b9b147d35c4fc3e\": rpc error: code = NotFound desc = could not find container \"c7a2029d6716d6fc1da85e0bb5f9d6b6f126d30282464fb15b9b147d35c4fc3e\": container with ID starting with c7a2029d6716d6fc1da85e0bb5f9d6b6f126d30282464fb15b9b147d35c4fc3e not found: ID does not exist"
Sep 14 08:12:39.671039 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.671021    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"]
Sep 14 08:12:39.673756 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.673738    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b914f7-7fdcdf7f74-dflmn"]
Sep 14 08:12:39.682184 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.682163    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"]
Sep 14 08:12:39.684465 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.684444    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b914f7-7b7f45fddf-w4ngw"]
Sep 14 08:12:39.886216 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.886184    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="314e4541-8302-47e9-92ad-3a5df13bcd36" path="/var/lib/kubelet/pods/314e4541-8302-47e9-92ad-3a5df13bcd36/volumes"
Sep 14 08:12:39.886562 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.886549    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4688740a-752c-4185-8ee0-28a16b105294" path="/var/lib/kubelet/pods/4688740a-752c-4185-8ee0-28a16b105294/volumes"
Sep 14 08:12:39.886907 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:39.886894    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77e81330-0faa-496f-a4bd-32db29f089fb" path="/var/lib/kubelet/pods/77e81330-0faa-496f-a4bd-32db29f089fb/volumes"
Sep 14 08:12:45.836339 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836305    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w"]
Sep 14 08:12:45.836824 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836810    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="314e4541-8302-47e9-92ad-3a5df13bcd36" containerName="revoke-keys"
Sep 14 08:12:45.836875 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836828    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="314e4541-8302-47e9-92ad-3a5df13bcd36" containerName="revoke-keys"
Sep 14 08:12:45.836875 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836840    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4688740a-752c-4185-8ee0-28a16b105294" containerName="payload-pre-processing"
Sep 14 08:12:45.836875 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836846    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4688740a-752c-4185-8ee0-28a16b105294" containerName="payload-pre-processing"
Sep 14 08:12:45.836875 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836857    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab0c3b13-6781-4162-ac9c-85416ac28bae" containerName="payload-processing"
Sep 14 08:12:45.836875 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836864    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab0c3b13-6781-4162-ac9c-85416ac28bae" containerName="payload-processing"
Sep 14 08:12:45.836875 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836876    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66336fc7-cbac-4183-afbf-d8e005cdcbbb" containerName="payload-pre-processing"
Sep 14 08:12:45.837055 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836881    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="66336fc7-cbac-4183-afbf-d8e005cdcbbb" containerName="payload-pre-processing"
Sep 14 08:12:45.837055 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836889    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77e81330-0faa-496f-a4bd-32db29f089fb" containerName="payload-processing"
Sep 14 08:12:45.837055 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836895    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="77e81330-0faa-496f-a4bd-32db29f089fb" containerName="payload-processing"
Sep 14 08:12:45.837055 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836959    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4688740a-752c-4185-8ee0-28a16b105294" containerName="payload-pre-processing"
Sep 14 08:12:45.837055 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836971    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="66336fc7-cbac-4183-afbf-d8e005cdcbbb" containerName="payload-pre-processing"
Sep 14 08:12:45.837055 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836979    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="77e81330-0faa-496f-a4bd-32db29f089fb" containerName="payload-processing"
Sep 14 08:12:45.837055 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836985    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="314e4541-8302-47e9-92ad-3a5df13bcd36" containerName="revoke-keys"
Sep 14 08:12:45.837055 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.836991    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab0c3b13-6781-4162-ac9c-85416ac28bae" containerName="payload-processing"
Sep 14 08:12:45.841888 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.841861    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w"
Sep 14 08:12:45.844945 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.844710    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:12:45.850182 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.850158    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w"]
Sep 14 08:12:45.963182 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:45.963148    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a6d986-d216-4d8d-b339-b694f2bbdb02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w\" (UID: \"19a6d986-d216-4d8d-b339-b694f2bbdb02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w"
Sep 14 08:12:46.064480 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:46.064447    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a6d986-d216-4d8d-b339-b694f2bbdb02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w\" (UID: \"19a6d986-d216-4d8d-b339-b694f2bbdb02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w"
Sep 14 08:12:46.065075 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:46.065053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a6d986-d216-4d8d-b339-b694f2bbdb02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w\" (UID: \"19a6d986-d216-4d8d-b339-b694f2bbdb02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w"
Sep 14 08:12:46.157430 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:46.157346    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w"
Sep 14 08:12:46.429037 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:46.429011    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w"]
Sep 14 08:12:46.429875 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:12:46.429847    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a6d986_d216_4d8d_b339_b694f2bbdb02.slice/crio-181a1f799f9419c02162d53e37284127fc6e9c16150d52493b9b86e951f3f99f WatchSource:0}: Error finding container 181a1f799f9419c02162d53e37284127fc6e9c16150d52493b9b86e951f3f99f: Status 404 returned error can't find the container with id 181a1f799f9419c02162d53e37284127fc6e9c16150d52493b9b86e951f3f99f
Sep 14 08:12:46.695480 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:46.695395    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w" event={"ID":"19a6d986-d216-4d8d-b339-b694f2bbdb02","Type":"ContainerStarted","Data":"181a1f799f9419c02162d53e37284127fc6e9c16150d52493b9b86e951f3f99f"}
Sep 14 08:12:47.704341 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:47.704264    2563 generic.go:356] "Generic (PLEG): container finished" podID="19a6d986-d216-4d8d-b339-b694f2bbdb02" containerID="95a765229243964d2753729479bf07fc13cdc5747cd52c45517519e0758178b8" exitCode=0
Sep 14 08:12:47.704341 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:47.704316    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w" event={"ID":"19a6d986-d216-4d8d-b339-b694f2bbdb02","Type":"ContainerDied","Data":"95a765229243964d2753729479bf07fc13cdc5747cd52c45517519e0758178b8"}
Sep 14 08:12:48.860092 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:48.860068    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w"
Sep 14 08:12:48.995587 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:48.995509    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a6d986-d216-4d8d-b339-b694f2bbdb02-maas-api-service-ca\") pod \"19a6d986-d216-4d8d-b339-b694f2bbdb02\" (UID: \"19a6d986-d216-4d8d-b339-b694f2bbdb02\") "
Sep 14 08:12:48.995953 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:48.995930    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19a6d986-d216-4d8d-b339-b694f2bbdb02-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "19a6d986-d216-4d8d-b339-b694f2bbdb02" (UID: "19a6d986-d216-4d8d-b339-b694f2bbdb02"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:49.096547 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:49.096515    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19a6d986-d216-4d8d-b339-b694f2bbdb02-maas-api-service-ca\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:12:49.726004 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:49.725971    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w"
Sep 14 08:12:49.726004 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:49.725980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w" event={"ID":"19a6d986-d216-4d8d-b339-b694f2bbdb02","Type":"ContainerDied","Data":"181a1f799f9419c02162d53e37284127fc6e9c16150d52493b9b86e951f3f99f"}
Sep 14 08:12:49.726224 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:49.726058    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="181a1f799f9419c02162d53e37284127fc6e9c16150d52493b9b86e951f3f99f"
Sep 14 08:12:59.926915 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:59.926885    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 08:12:59.932488 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:12:59.932466    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 08:13:01.685352 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:01.685317    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"]
Sep 14 08:13:01.685816 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:01.685575    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg" podUID="edc3aa79-bb42-4a98-b462-932104c07280" containerName="payload-processing" containerID="cri-o://f22c7fd4d877281d2e37cb793964c8874edebaef30cba2efd9fb7d9a81098f19" gracePeriod=30
Sep 14 08:13:01.803272 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:01.803236    2563 generic.go:356] "Generic (PLEG): container finished" podID="edc3aa79-bb42-4a98-b462-932104c07280" containerID="f22c7fd4d877281d2e37cb793964c8874edebaef30cba2efd9fb7d9a81098f19" exitCode=0
Sep 14 08:13:01.803413 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:01.803308    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg" event={"ID":"edc3aa79-bb42-4a98-b462-932104c07280","Type":"ContainerDied","Data":"f22c7fd4d877281d2e37cb793964c8874edebaef30cba2efd9fb7d9a81098f19"}
Sep 14 08:13:01.860005 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:01.859959    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w"]
Sep 14 08:13:01.864499 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:01.864466    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-87b7cb-q7n5w"]
Sep 14 08:13:01.888858 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:01.888822    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19a6d986-d216-4d8d-b339-b694f2bbdb02" path="/var/lib/kubelet/pods/19a6d986-d216-4d8d-b339-b694f2bbdb02/volumes"
Sep 14 08:13:01.952557 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:01.952472    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"
Sep 14 08:13:02.121164 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:02.121133    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9c75\" (UniqueName: \"kubernetes.io/projected/edc3aa79-bb42-4a98-b462-932104c07280-kube-api-access-j9c75\") pod \"edc3aa79-bb42-4a98-b462-932104c07280\" (UID: \"edc3aa79-bb42-4a98-b462-932104c07280\") "
Sep 14 08:13:02.121367 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:02.121242    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edc3aa79-bb42-4a98-b462-932104c07280-plugins-config-volume\") pod \"edc3aa79-bb42-4a98-b462-932104c07280\" (UID: \"edc3aa79-bb42-4a98-b462-932104c07280\") "
Sep 14 08:13:02.121583 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:02.121560    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edc3aa79-bb42-4a98-b462-932104c07280-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "edc3aa79-bb42-4a98-b462-932104c07280" (UID: "edc3aa79-bb42-4a98-b462-932104c07280"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:02.123173 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:02.123151    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edc3aa79-bb42-4a98-b462-932104c07280-kube-api-access-j9c75" (OuterVolumeSpecName: "kube-api-access-j9c75") pod "edc3aa79-bb42-4a98-b462-932104c07280" (UID: "edc3aa79-bb42-4a98-b462-932104c07280"). InnerVolumeSpecName "kube-api-access-j9c75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:02.222836 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:02.222753    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9c75\" (UniqueName: \"kubernetes.io/projected/edc3aa79-bb42-4a98-b462-932104c07280-kube-api-access-j9c75\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:02.222836 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:02.222783    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edc3aa79-bb42-4a98-b462-932104c07280-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:02.811209 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:02.811168    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg" event={"ID":"edc3aa79-bb42-4a98-b462-932104c07280","Type":"ContainerDied","Data":"5b44ab939dacc440c5083e1cceb9c0e6cb18b114a692c39ffe187d46aa134a3b"}
Sep 14 08:13:02.811209 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:02.811203    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"
Sep 14 08:13:02.811749 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:02.811243    2563 scope.go:117] "RemoveContainer" containerID="f22c7fd4d877281d2e37cb793964c8874edebaef30cba2efd9fb7d9a81098f19"
Sep 14 08:13:02.833688 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:02.833662    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"]
Sep 14 08:13:02.836136 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:02.836114    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-87b7cb-66f887ff99-n94qg"]
Sep 14 08:13:03.887416 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:03.887377    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edc3aa79-bb42-4a98-b462-932104c07280" path="/var/lib/kubelet/pods/edc3aa79-bb42-4a98-b462-932104c07280/volumes"
Sep 14 08:13:19.627092 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.627051    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"]
Sep 14 08:13:19.627581 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.627352    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr" podUID="8e20d590-36f5-443f-838f-6696ccefd7a3" containerName="payload-processing" containerID="cri-o://e7d13bb37bc4ffdb7a3179be47c78f81f1dafff9101ecde90d0cb8531a07fd72" gracePeriod=30
Sep 14 08:13:19.655121 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.655087    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"]
Sep 14 08:13:19.655655 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.655588    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt" podUID="b54011f6-c356-408a-bb06-41d0e43121e9" containerName="payload-pre-processing" containerID="cri-o://59115d7f3c5628f941bf25ca92f8c802fca67fe4d95a4657b8e314f359e3764a" gracePeriod=30
Sep 14 08:13:19.926036 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.926004    2563 generic.go:356] "Generic (PLEG): container finished" podID="8e20d590-36f5-443f-838f-6696ccefd7a3" containerID="e7d13bb37bc4ffdb7a3179be47c78f81f1dafff9101ecde90d0cb8531a07fd72" exitCode=0
Sep 14 08:13:19.926170 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.926085    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr" event={"ID":"8e20d590-36f5-443f-838f-6696ccefd7a3","Type":"ContainerDied","Data":"e7d13bb37bc4ffdb7a3179be47c78f81f1dafff9101ecde90d0cb8531a07fd72"}
Sep 14 08:13:19.927993 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.927970    2563 generic.go:356] "Generic (PLEG): container finished" podID="b54011f6-c356-408a-bb06-41d0e43121e9" containerID="59115d7f3c5628f941bf25ca92f8c802fca67fe4d95a4657b8e314f359e3764a" exitCode=0
Sep 14 08:13:19.928102 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.928000    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt" event={"ID":"b54011f6-c356-408a-bb06-41d0e43121e9","Type":"ContainerDied","Data":"59115d7f3c5628f941bf25ca92f8c802fca67fe4d95a4657b8e314f359e3764a"}
Sep 14 08:13:19.961363 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.961336    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"
Sep 14 08:13:19.965230 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.965208    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"
Sep 14 08:13:19.991802 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.991768    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6rhq\" (UniqueName: \"kubernetes.io/projected/b54011f6-c356-408a-bb06-41d0e43121e9-kube-api-access-g6rhq\") pod \"b54011f6-c356-408a-bb06-41d0e43121e9\" (UID: \"b54011f6-c356-408a-bb06-41d0e43121e9\") "
Sep 14 08:13:19.991924 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.991867    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e20d590-36f5-443f-838f-6696ccefd7a3-plugins-config-volume\") pod \"8e20d590-36f5-443f-838f-6696ccefd7a3\" (UID: \"8e20d590-36f5-443f-838f-6696ccefd7a3\") "
Sep 14 08:13:19.991990 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.991937    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b54011f6-c356-408a-bb06-41d0e43121e9-plugins-config-volume\") pod \"b54011f6-c356-408a-bb06-41d0e43121e9\" (UID: \"b54011f6-c356-408a-bb06-41d0e43121e9\") "
Sep 14 08:13:19.992051 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.992007    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zx8mt\" (UniqueName: \"kubernetes.io/projected/8e20d590-36f5-443f-838f-6696ccefd7a3-kube-api-access-zx8mt\") pod \"8e20d590-36f5-443f-838f-6696ccefd7a3\" (UID: \"8e20d590-36f5-443f-838f-6696ccefd7a3\") "
Sep 14 08:13:19.992261 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.992236    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b54011f6-c356-408a-bb06-41d0e43121e9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b54011f6-c356-408a-bb06-41d0e43121e9" (UID: "b54011f6-c356-408a-bb06-41d0e43121e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:19.992359 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.992237    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e20d590-36f5-443f-838f-6696ccefd7a3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e20d590-36f5-443f-838f-6696ccefd7a3" (UID: "8e20d590-36f5-443f-838f-6696ccefd7a3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:19.992424 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.992382    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e20d590-36f5-443f-838f-6696ccefd7a3-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:19.992424 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.992397    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b54011f6-c356-408a-bb06-41d0e43121e9-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:19.994259 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.994225    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e20d590-36f5-443f-838f-6696ccefd7a3-kube-api-access-zx8mt" (OuterVolumeSpecName: "kube-api-access-zx8mt") pod "8e20d590-36f5-443f-838f-6696ccefd7a3" (UID: "8e20d590-36f5-443f-838f-6696ccefd7a3"). InnerVolumeSpecName "kube-api-access-zx8mt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:19.995136 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:19.994949    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b54011f6-c356-408a-bb06-41d0e43121e9-kube-api-access-g6rhq" (OuterVolumeSpecName: "kube-api-access-g6rhq") pod "b54011f6-c356-408a-bb06-41d0e43121e9" (UID: "b54011f6-c356-408a-bb06-41d0e43121e9"). InnerVolumeSpecName "kube-api-access-g6rhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:20.093722 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:20.093688    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zx8mt\" (UniqueName: \"kubernetes.io/projected/8e20d590-36f5-443f-838f-6696ccefd7a3-kube-api-access-zx8mt\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:20.093722 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:20.093718    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6rhq\" (UniqueName: \"kubernetes.io/projected/b54011f6-c356-408a-bb06-41d0e43121e9-kube-api-access-g6rhq\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:20.935459 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:20.935432    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"
Sep 14 08:13:20.935888 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:20.935468    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt" event={"ID":"b54011f6-c356-408a-bb06-41d0e43121e9","Type":"ContainerDied","Data":"66ae9a0568b36f59c6886c0b208275b9914ed6e408daac9d13674c9078ef31d9"}
Sep 14 08:13:20.935888 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:20.935529    2563 scope.go:117] "RemoveContainer" containerID="59115d7f3c5628f941bf25ca92f8c802fca67fe4d95a4657b8e314f359e3764a"
Sep 14 08:13:20.937249 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:20.937231    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"
Sep 14 08:13:20.937249 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:20.937236    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr" event={"ID":"8e20d590-36f5-443f-838f-6696ccefd7a3","Type":"ContainerDied","Data":"5a8234b4c1284aabda15846735d83bb8b1be64b927e830addc2f4fc5e9410532"}
Sep 14 08:13:20.946665 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:20.946623    2563 scope.go:117] "RemoveContainer" containerID="e7d13bb37bc4ffdb7a3179be47c78f81f1dafff9101ecde90d0cb8531a07fd72"
Sep 14 08:13:20.961980 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:20.961959    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"]
Sep 14 08:13:20.966029 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:20.966009    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b990f4-6f9d566d4b-nc9nt"]
Sep 14 08:13:20.975467 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:20.975445    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"]
Sep 14 08:13:20.979145 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:20.979125    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b990f4-5cc64fccb5-vm8nr"]
Sep 14 08:13:21.886653 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:21.886601    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e20d590-36f5-443f-838f-6696ccefd7a3" path="/var/lib/kubelet/pods/8e20d590-36f5-443f-838f-6696ccefd7a3/volumes"
Sep 14 08:13:21.886996 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:21.886982    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b54011f6-c356-408a-bb06-41d0e43121e9" path="/var/lib/kubelet/pods/b54011f6-c356-408a-bb06-41d0e43121e9/volumes"
Sep 14 08:13:22.084508 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:22.084425    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"]
Sep 14 08:13:37.104402 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.104354    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9" podUID="f05750e9-c618-4aba-b38f-995682723fd9" containerName="main" containerID="cri-o://d50d5508615854849d0350f4fdfd217d1fb1a5fbefb3e4798448425f532d3a9f" gracePeriod=45
Sep 14 08:13:37.271571 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.271532    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"]
Sep 14 08:13:37.271839 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.271807    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn" podUID="09bd8e46-b966-401a-abac-7ef008d8c051" containerName="payload-processing" containerID="cri-o://d0acdb082eae2a14d3da10da7e8bc29f8187417351a9397411e3c7a57f896907" gracePeriod=30
Sep 14 08:13:37.277586 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.277557    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"]
Sep 14 08:13:37.279213 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.279184    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw" podUID="86f0e8a6-58bd-46d0-9003-1d9d113d22a8" containerName="payload-pre-processing" containerID="cri-o://72c6abfafae5df1b572c3cd58101157315ae20bf96a975bc55c2f4d15bfd914a" gracePeriod=30
Sep 14 08:13:37.370310 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.370225    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn" podUID="09bd8e46-b966-401a-abac-7ef008d8c051" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.77:9004: connect: connection refused"
Sep 14 08:13:37.370479 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.370228    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw" podUID="86f0e8a6-58bd-46d0-9003-1d9d113d22a8" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.76:9004: connect: connection refused"
Sep 14 08:13:37.430066 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.429983    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:13:37.556455 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.556425    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-model-cache\") pod \"f05750e9-c618-4aba-b38f-995682723fd9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") "
Sep 14 08:13:37.556584 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.556529    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f05750e9-c618-4aba-b38f-995682723fd9-tls-certs\") pod \"f05750e9-c618-4aba-b38f-995682723fd9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") "
Sep 14 08:13:37.556584 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.556573    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-tmp-dir\") pod \"f05750e9-c618-4aba-b38f-995682723fd9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") "
Sep 14 08:13:37.556734 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.556607    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6tkx\" (UniqueName: \"kubernetes.io/projected/f05750e9-c618-4aba-b38f-995682723fd9-kube-api-access-x6tkx\") pod \"f05750e9-c618-4aba-b38f-995682723fd9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") "
Sep 14 08:13:37.556734 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.556665    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-home\") pod \"f05750e9-c618-4aba-b38f-995682723fd9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") "
Sep 14 08:13:37.556734 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.556698    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-dshm\") pod \"f05750e9-c618-4aba-b38f-995682723fd9\" (UID: \"f05750e9-c618-4aba-b38f-995682723fd9\") "
Sep 14 08:13:37.556734 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.556701    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-model-cache" (OuterVolumeSpecName: "model-cache") pod "f05750e9-c618-4aba-b38f-995682723fd9" (UID: "f05750e9-c618-4aba-b38f-995682723fd9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:37.557016 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.556993    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-home" (OuterVolumeSpecName: "home") pod "f05750e9-c618-4aba-b38f-995682723fd9" (UID: "f05750e9-c618-4aba-b38f-995682723fd9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:37.557102 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.557005    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-model-cache\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:37.557102 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.557085    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f05750e9-c618-4aba-b38f-995682723fd9" (UID: "f05750e9-c618-4aba-b38f-995682723fd9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:37.559104 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.559074    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f05750e9-c618-4aba-b38f-995682723fd9-kube-api-access-x6tkx" (OuterVolumeSpecName: "kube-api-access-x6tkx") pod "f05750e9-c618-4aba-b38f-995682723fd9" (UID: "f05750e9-c618-4aba-b38f-995682723fd9"). InnerVolumeSpecName "kube-api-access-x6tkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:37.559221 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.559159    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-dshm" (OuterVolumeSpecName: "dshm") pod "f05750e9-c618-4aba-b38f-995682723fd9" (UID: "f05750e9-c618-4aba-b38f-995682723fd9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:37.559221 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.559188    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f05750e9-c618-4aba-b38f-995682723fd9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f05750e9-c618-4aba-b38f-995682723fd9" (UID: "f05750e9-c618-4aba-b38f-995682723fd9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:13:37.563568 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.563545    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"
Sep 14 08:13:37.575834 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.575812    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"
Sep 14 08:13:37.658218 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.658130    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86f0e8a6-58bd-46d0-9003-1d9d113d22a8-plugins-config-volume\") pod \"86f0e8a6-58bd-46d0-9003-1d9d113d22a8\" (UID: \"86f0e8a6-58bd-46d0-9003-1d9d113d22a8\") "
Sep 14 08:13:37.658218 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.658189    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdv4p\" (UniqueName: \"kubernetes.io/projected/09bd8e46-b966-401a-abac-7ef008d8c051-kube-api-access-mdv4p\") pod \"09bd8e46-b966-401a-abac-7ef008d8c051\" (UID: \"09bd8e46-b966-401a-abac-7ef008d8c051\") "
Sep 14 08:13:37.658218 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.658216    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvz4d\" (UniqueName: \"kubernetes.io/projected/86f0e8a6-58bd-46d0-9003-1d9d113d22a8-kube-api-access-xvz4d\") pod \"86f0e8a6-58bd-46d0-9003-1d9d113d22a8\" (UID: \"86f0e8a6-58bd-46d0-9003-1d9d113d22a8\") "
Sep 14 08:13:37.658520 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.658242    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09bd8e46-b966-401a-abac-7ef008d8c051-plugins-config-volume\") pod \"09bd8e46-b966-401a-abac-7ef008d8c051\" (UID: \"09bd8e46-b966-401a-abac-7ef008d8c051\") "
Sep 14 08:13:37.658588 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.658557    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f05750e9-c618-4aba-b38f-995682723fd9-tls-certs\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:37.658588 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.658576    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-tmp-dir\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:37.658701 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.658597    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6tkx\" (UniqueName: \"kubernetes.io/projected/f05750e9-c618-4aba-b38f-995682723fd9-kube-api-access-x6tkx\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:37.658701 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.658596    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86f0e8a6-58bd-46d0-9003-1d9d113d22a8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "86f0e8a6-58bd-46d0-9003-1d9d113d22a8" (UID: "86f0e8a6-58bd-46d0-9003-1d9d113d22a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:37.658701 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.658614    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-home\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:37.658701 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.658693    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f05750e9-c618-4aba-b38f-995682723fd9-dshm\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:37.658851 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.658614    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09bd8e46-b966-401a-abac-7ef008d8c051-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "09bd8e46-b966-401a-abac-7ef008d8c051" (UID: "09bd8e46-b966-401a-abac-7ef008d8c051"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:37.660376 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.660354    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86f0e8a6-58bd-46d0-9003-1d9d113d22a8-kube-api-access-xvz4d" (OuterVolumeSpecName: "kube-api-access-xvz4d") pod "86f0e8a6-58bd-46d0-9003-1d9d113d22a8" (UID: "86f0e8a6-58bd-46d0-9003-1d9d113d22a8"). InnerVolumeSpecName "kube-api-access-xvz4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:37.660473 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.660384    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09bd8e46-b966-401a-abac-7ef008d8c051-kube-api-access-mdv4p" (OuterVolumeSpecName: "kube-api-access-mdv4p") pod "09bd8e46-b966-401a-abac-7ef008d8c051" (UID: "09bd8e46-b966-401a-abac-7ef008d8c051"). InnerVolumeSpecName "kube-api-access-mdv4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:37.760105 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.760068    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdv4p\" (UniqueName: \"kubernetes.io/projected/09bd8e46-b966-401a-abac-7ef008d8c051-kube-api-access-mdv4p\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:37.760105 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.760098    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvz4d\" (UniqueName: \"kubernetes.io/projected/86f0e8a6-58bd-46d0-9003-1d9d113d22a8-kube-api-access-xvz4d\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:37.760105 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.760114    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09bd8e46-b966-401a-abac-7ef008d8c051-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:37.760329 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:37.760126    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86f0e8a6-58bd-46d0-9003-1d9d113d22a8-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:13:38.055045 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.054953    2563 generic.go:356] "Generic (PLEG): container finished" podID="09bd8e46-b966-401a-abac-7ef008d8c051" containerID="d0acdb082eae2a14d3da10da7e8bc29f8187417351a9397411e3c7a57f896907" exitCode=0
Sep 14 08:13:38.055045 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.055012    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"
Sep 14 08:13:38.055293 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.055038    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn" event={"ID":"09bd8e46-b966-401a-abac-7ef008d8c051","Type":"ContainerDied","Data":"d0acdb082eae2a14d3da10da7e8bc29f8187417351a9397411e3c7a57f896907"}
Sep 14 08:13:38.055293 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.055100    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn" event={"ID":"09bd8e46-b966-401a-abac-7ef008d8c051","Type":"ContainerDied","Data":"0ae2eae4ad320aa160e3f6afe2fdf9db32b0ec6a4415e7a3906b96900a4b9eb6"}
Sep 14 08:13:38.055293 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.055134    2563 scope.go:117] "RemoveContainer" containerID="d0acdb082eae2a14d3da10da7e8bc29f8187417351a9397411e3c7a57f896907"
Sep 14 08:13:38.056887 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.056860    2563 generic.go:356] "Generic (PLEG): container finished" podID="f05750e9-c618-4aba-b38f-995682723fd9" containerID="d50d5508615854849d0350f4fdfd217d1fb1a5fbefb3e4798448425f532d3a9f" exitCode=0
Sep 14 08:13:38.057002 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.056947    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9" event={"ID":"f05750e9-c618-4aba-b38f-995682723fd9","Type":"ContainerDied","Data":"d50d5508615854849d0350f4fdfd217d1fb1a5fbefb3e4798448425f532d3a9f"}
Sep 14 08:13:38.057002 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.056978    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9" event={"ID":"f05750e9-c618-4aba-b38f-995682723fd9","Type":"ContainerDied","Data":"d342efa6a2793e513fb8f285810f7fefd633d9d5a588ff495abd968bf4030981"}
Sep 14 08:13:38.057002 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.056994    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"
Sep 14 08:13:38.058721 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.058696    2563 generic.go:356] "Generic (PLEG): container finished" podID="86f0e8a6-58bd-46d0-9003-1d9d113d22a8" containerID="72c6abfafae5df1b572c3cd58101157315ae20bf96a975bc55c2f4d15bfd914a" exitCode=0
Sep 14 08:13:38.058824 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.058791    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"
Sep 14 08:13:38.058891 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.058819    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw" event={"ID":"86f0e8a6-58bd-46d0-9003-1d9d113d22a8","Type":"ContainerDied","Data":"72c6abfafae5df1b572c3cd58101157315ae20bf96a975bc55c2f4d15bfd914a"}
Sep 14 08:13:38.058891 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.058864    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw" event={"ID":"86f0e8a6-58bd-46d0-9003-1d9d113d22a8","Type":"ContainerDied","Data":"f7f2d2ba44fdf5c733ca4ea6a80d6a3372ef4125af135b9b21366ac84b0cec63"}
Sep 14 08:13:38.066085 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.066070    2563 scope.go:117] "RemoveContainer" containerID="d0acdb082eae2a14d3da10da7e8bc29f8187417351a9397411e3c7a57f896907"
Sep 14 08:13:38.066396 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:13:38.066379    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0acdb082eae2a14d3da10da7e8bc29f8187417351a9397411e3c7a57f896907\": container with ID starting with d0acdb082eae2a14d3da10da7e8bc29f8187417351a9397411e3c7a57f896907 not found: ID does not exist" containerID="d0acdb082eae2a14d3da10da7e8bc29f8187417351a9397411e3c7a57f896907"
Sep 14 08:13:38.066472 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.066406    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0acdb082eae2a14d3da10da7e8bc29f8187417351a9397411e3c7a57f896907"} err="failed to get container status \"d0acdb082eae2a14d3da10da7e8bc29f8187417351a9397411e3c7a57f896907\": rpc error: code = NotFound desc = could not find container \"d0acdb082eae2a14d3da10da7e8bc29f8187417351a9397411e3c7a57f896907\": container with ID starting with d0acdb082eae2a14d3da10da7e8bc29f8187417351a9397411e3c7a57f896907 not found: ID does not exist"
Sep 14 08:13:38.066472 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.066429    2563 scope.go:117] "RemoveContainer" containerID="d50d5508615854849d0350f4fdfd217d1fb1a5fbefb3e4798448425f532d3a9f"
Sep 14 08:13:38.076751 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.076730    2563 scope.go:117] "RemoveContainer" containerID="d50d5508615854849d0350f4fdfd217d1fb1a5fbefb3e4798448425f532d3a9f"
Sep 14 08:13:38.076852 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.076834    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"]
Sep 14 08:13:38.077156 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:13:38.077131    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d50d5508615854849d0350f4fdfd217d1fb1a5fbefb3e4798448425f532d3a9f\": container with ID starting with d50d5508615854849d0350f4fdfd217d1fb1a5fbefb3e4798448425f532d3a9f not found: ID does not exist" containerID="d50d5508615854849d0350f4fdfd217d1fb1a5fbefb3e4798448425f532d3a9f"
Sep 14 08:13:38.077306 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.077284    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50d5508615854849d0350f4fdfd217d1fb1a5fbefb3e4798448425f532d3a9f"} err="failed to get container status \"d50d5508615854849d0350f4fdfd217d1fb1a5fbefb3e4798448425f532d3a9f\": rpc error: code = NotFound desc = could not find container \"d50d5508615854849d0350f4fdfd217d1fb1a5fbefb3e4798448425f532d3a9f\": container with ID starting with d50d5508615854849d0350f4fdfd217d1fb1a5fbefb3e4798448425f532d3a9f not found: ID does not exist"
Sep 14 08:13:38.077415 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.077401    2563 scope.go:117] "RemoveContainer" containerID="72c6abfafae5df1b572c3cd58101157315ae20bf96a975bc55c2f4d15bfd914a"
Sep 14 08:13:38.080990 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.080972    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-afcf8a-7f47db574f-n2wgn"]
Sep 14 08:13:38.086152 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.086123    2563 scope.go:117] "RemoveContainer" containerID="72c6abfafae5df1b572c3cd58101157315ae20bf96a975bc55c2f4d15bfd914a"
Sep 14 08:13:38.086408 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:13:38.086389    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72c6abfafae5df1b572c3cd58101157315ae20bf96a975bc55c2f4d15bfd914a\": container with ID starting with 72c6abfafae5df1b572c3cd58101157315ae20bf96a975bc55c2f4d15bfd914a not found: ID does not exist" containerID="72c6abfafae5df1b572c3cd58101157315ae20bf96a975bc55c2f4d15bfd914a"
Sep 14 08:13:38.086466 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.086415    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72c6abfafae5df1b572c3cd58101157315ae20bf96a975bc55c2f4d15bfd914a"} err="failed to get container status \"72c6abfafae5df1b572c3cd58101157315ae20bf96a975bc55c2f4d15bfd914a\": rpc error: code = NotFound desc = could not find container \"72c6abfafae5df1b572c3cd58101157315ae20bf96a975bc55c2f4d15bfd914a\": container with ID starting with 72c6abfafae5df1b572c3cd58101157315ae20bf96a975bc55c2f4d15bfd914a not found: ID does not exist"
Sep 14 08:13:38.092919 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.092898    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"]
Sep 14 08:13:38.097184 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.097158    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/auth-test-model-ea9d35-kserve-9b64cc49f-gc4m9"]
Sep 14 08:13:38.107370 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.107350    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"]
Sep 14 08:13:38.113918 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:38.113900    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-afcf8a-6cdf6db94b-vlzbw"]
Sep 14 08:13:39.887443 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:39.887409    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09bd8e46-b966-401a-abac-7ef008d8c051" path="/var/lib/kubelet/pods/09bd8e46-b966-401a-abac-7ef008d8c051/volumes"
Sep 14 08:13:39.887838 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:39.887790    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86f0e8a6-58bd-46d0-9003-1d9d113d22a8" path="/var/lib/kubelet/pods/86f0e8a6-58bd-46d0-9003-1d9d113d22a8/volumes"
Sep 14 08:13:39.888165 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:13:39.888152    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f05750e9-c618-4aba-b38f-995682723fd9" path="/var/lib/kubelet/pods/f05750e9-c618-4aba-b38f-995682723fd9/volumes"
Sep 14 08:14:00.132900 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:14:00.132870    2563 scope.go:117] "RemoveContainer" containerID="e46a55a9898c40b57846fa3cbfd9429497d1ef56b0e3ae1e8ac3cf833966d0f8"
Sep 14 08:14:00.141539 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:14:00.141509    2563 scope.go:117] "RemoveContainer" containerID="1c405cb6a3ac9c55abff7b1329947981fd8d4149490055b0f45065c24e93150c"
Sep 14 08:15:00.144036 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.143954    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ea9d35-29822895-8mx2s"]
Sep 14 08:15:00.144527 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144512    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09bd8e46-b966-401a-abac-7ef008d8c051" containerName="payload-processing"
Sep 14 08:15:00.144572 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144532    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="09bd8e46-b966-401a-abac-7ef008d8c051" containerName="payload-processing"
Sep 14 08:15:00.144572 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144547    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edc3aa79-bb42-4a98-b462-932104c07280" containerName="payload-processing"
Sep 14 08:15:00.144572 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144556    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="edc3aa79-bb42-4a98-b462-932104c07280" containerName="payload-processing"
Sep 14 08:15:00.144694 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144572    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b54011f6-c356-408a-bb06-41d0e43121e9" containerName="payload-pre-processing"
Sep 14 08:15:00.144694 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144581    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b54011f6-c356-408a-bb06-41d0e43121e9" containerName="payload-pre-processing"
Sep 14 08:15:00.144694 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144600    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19a6d986-d216-4d8d-b339-b694f2bbdb02" containerName="revoke-keys"
Sep 14 08:15:00.144694 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144608    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a6d986-d216-4d8d-b339-b694f2bbdb02" containerName="revoke-keys"
Sep 14 08:15:00.144694 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144631    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86f0e8a6-58bd-46d0-9003-1d9d113d22a8" containerName="payload-pre-processing"
Sep 14 08:15:00.144694 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144654    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="86f0e8a6-58bd-46d0-9003-1d9d113d22a8" containerName="payload-pre-processing"
Sep 14 08:15:00.144694 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144666    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e20d590-36f5-443f-838f-6696ccefd7a3" containerName="payload-processing"
Sep 14 08:15:00.144694 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144674    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e20d590-36f5-443f-838f-6696ccefd7a3" containerName="payload-processing"
Sep 14 08:15:00.144694 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144684    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f05750e9-c618-4aba-b38f-995682723fd9" containerName="main"
Sep 14 08:15:00.144694 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144692    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f05750e9-c618-4aba-b38f-995682723fd9" containerName="main"
Sep 14 08:15:00.145008 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144776    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="09bd8e46-b966-401a-abac-7ef008d8c051" containerName="payload-processing"
Sep 14 08:15:00.145008 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144789    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b54011f6-c356-408a-bb06-41d0e43121e9" containerName="payload-pre-processing"
Sep 14 08:15:00.145008 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144798    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="19a6d986-d216-4d8d-b339-b694f2bbdb02" containerName="revoke-keys"
Sep 14 08:15:00.145008 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144807    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="86f0e8a6-58bd-46d0-9003-1d9d113d22a8" containerName="payload-pre-processing"
Sep 14 08:15:00.145008 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144818    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="edc3aa79-bb42-4a98-b462-932104c07280" containerName="payload-processing"
Sep 14 08:15:00.145008 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144827    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e20d590-36f5-443f-838f-6696ccefd7a3" containerName="payload-processing"
Sep 14 08:15:00.145008 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.144835    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f05750e9-c618-4aba-b38f-995682723fd9" containerName="main"
Sep 14 08:15:00.148504 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.148480    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ea9d35-29822895-8mx2s"
Sep 14 08:15:00.151196 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.151170    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:15:00.152922 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.152899    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ea9d35-29822895-8mx2s"]
Sep 14 08:15:00.162167 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.162149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ea9d35-29822895-8mx2s"
Sep 14 08:15:00.434332 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.434295    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ea9d35-29822895-8mx2s"]
Sep 14 08:15:00.435431 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:15:00.435408    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf475afb8_f976_439d_aa81_95563a8bfb7a.slice/crio-5c03880aa1594152e0e423f0f71e3380ee1b29c7b755821723c9e40f5db12445 WatchSource:0}: Error finding container 5c03880aa1594152e0e423f0f71e3380ee1b29c7b755821723c9e40f5db12445: Status 404 returned error can't find the container with id 5c03880aa1594152e0e423f0f71e3380ee1b29c7b755821723c9e40f5db12445
Sep 14 08:15:00.547296 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:00.547265    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ea9d35-29822895-8mx2s" event={"ID":"f475afb8-f976-439d-aa81-95563a8bfb7a","Type":"ContainerStarted","Data":"5c03880aa1594152e0e423f0f71e3380ee1b29c7b755821723c9e40f5db12445"}
Sep 14 08:15:01.559276 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:01.559234    2563 generic.go:356] "Generic (PLEG): container finished" podID="f475afb8-f976-439d-aa81-95563a8bfb7a" containerID="85b3e5a1266cf2ba5fef4ae594cc4fedf8afe2581cdfea76d63c1c9550550d2f" exitCode=0
Sep 14 08:15:01.559760 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:01.559325    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ea9d35-29822895-8mx2s" event={"ID":"f475afb8-f976-439d-aa81-95563a8bfb7a","Type":"ContainerDied","Data":"85b3e5a1266cf2ba5fef4ae594cc4fedf8afe2581cdfea76d63c1c9550550d2f"}
Sep 14 08:15:02.707326 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:02.707301    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ea9d35-29822895-8mx2s"
Sep 14 08:15:03.573363 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:03.573321    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ea9d35-29822895-8mx2s"
Sep 14 08:15:03.573363 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:03.573351    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ea9d35-29822895-8mx2s" event={"ID":"f475afb8-f976-439d-aa81-95563a8bfb7a","Type":"ContainerDied","Data":"5c03880aa1594152e0e423f0f71e3380ee1b29c7b755821723c9e40f5db12445"}
Sep 14 08:15:03.573589 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:15:03.573408    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c03880aa1594152e0e423f0f71e3380ee1b29c7b755821723c9e40f5db12445"
Sep 14 08:16:28.471424 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.471332    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"]
Sep 14 08:16:28.471912 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.471826    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f475afb8-f976-439d-aa81-95563a8bfb7a" containerName="cleanup"
Sep 14 08:16:28.471912 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.471841    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f475afb8-f976-439d-aa81-95563a8bfb7a" containerName="cleanup"
Sep 14 08:16:28.471912 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.471904    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f475afb8-f976-439d-aa81-95563a8bfb7a" containerName="cleanup"
Sep 14 08:16:28.475004 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.474985    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"
Sep 14 08:16:28.476980 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.476960    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-7f1623\""
Sep 14 08:16:28.477041 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.476991    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-7f1623-dockercfg-jtdnd\""
Sep 14 08:16:28.484940 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.484909    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"]
Sep 14 08:16:28.493401 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.493368    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"]
Sep 14 08:16:28.497126 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.497104    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"
Sep 14 08:16:28.508187 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.508160    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"]
Sep 14 08:16:28.570792 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.570758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcc72d3c-a579-42fa-bb7e-c75c42f99954-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7\" (UID: \"dcc72d3c-a579-42fa-bb7e-c75c42f99954\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"
Sep 14 08:16:28.570956 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.570802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9s88\" (UniqueName: \"kubernetes.io/projected/06917ca2-46da-44d6-8c3c-c89f251126fb-kube-api-access-h9s88\") pod \"payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr\" (UID: \"06917ca2-46da-44d6-8c3c-c89f251126fb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"
Sep 14 08:16:28.570956 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.570826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06917ca2-46da-44d6-8c3c-c89f251126fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr\" (UID: \"06917ca2-46da-44d6-8c3c-c89f251126fb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"
Sep 14 08:16:28.570956 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.570899    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49phx\" (UniqueName: \"kubernetes.io/projected/dcc72d3c-a579-42fa-bb7e-c75c42f99954-kube-api-access-49phx\") pod \"payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7\" (UID: \"dcc72d3c-a579-42fa-bb7e-c75c42f99954\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"
Sep 14 08:16:28.671766 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.671731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49phx\" (UniqueName: \"kubernetes.io/projected/dcc72d3c-a579-42fa-bb7e-c75c42f99954-kube-api-access-49phx\") pod \"payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7\" (UID: \"dcc72d3c-a579-42fa-bb7e-c75c42f99954\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"
Sep 14 08:16:28.671956 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.671803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcc72d3c-a579-42fa-bb7e-c75c42f99954-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7\" (UID: \"dcc72d3c-a579-42fa-bb7e-c75c42f99954\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"
Sep 14 08:16:28.671956 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.671832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9s88\" (UniqueName: \"kubernetes.io/projected/06917ca2-46da-44d6-8c3c-c89f251126fb-kube-api-access-h9s88\") pod \"payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr\" (UID: \"06917ca2-46da-44d6-8c3c-c89f251126fb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"
Sep 14 08:16:28.671956 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.671854    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06917ca2-46da-44d6-8c3c-c89f251126fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr\" (UID: \"06917ca2-46da-44d6-8c3c-c89f251126fb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"
Sep 14 08:16:28.672778 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.672751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06917ca2-46da-44d6-8c3c-c89f251126fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr\" (UID: \"06917ca2-46da-44d6-8c3c-c89f251126fb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"
Sep 14 08:16:28.672906 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.672751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcc72d3c-a579-42fa-bb7e-c75c42f99954-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7\" (UID: \"dcc72d3c-a579-42fa-bb7e-c75c42f99954\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"
Sep 14 08:16:28.679339 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.679316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9s88\" (UniqueName: \"kubernetes.io/projected/06917ca2-46da-44d6-8c3c-c89f251126fb-kube-api-access-h9s88\") pod \"payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr\" (UID: \"06917ca2-46da-44d6-8c3c-c89f251126fb\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"
Sep 14 08:16:28.679468 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.679318    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49phx\" (UniqueName: \"kubernetes.io/projected/dcc72d3c-a579-42fa-bb7e-c75c42f99954-kube-api-access-49phx\") pod \"payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7\" (UID: \"dcc72d3c-a579-42fa-bb7e-c75c42f99954\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"
Sep 14 08:16:28.785845 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.785778    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"
Sep 14 08:16:28.809255 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:28.809232    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"
Sep 14 08:16:29.061506 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:29.061466    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"]
Sep 14 08:16:29.061894 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:16:29.061872    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06917ca2_46da_44d6_8c3c_c89f251126fb.slice/crio-66f7e876f25ee206535400526cb3c2671693c201e2118a48f733b440299798b2 WatchSource:0}: Error finding container 66f7e876f25ee206535400526cb3c2671693c201e2118a48f733b440299798b2: Status 404 returned error can't find the container with id 66f7e876f25ee206535400526cb3c2671693c201e2118a48f733b440299798b2
Sep 14 08:16:29.069935 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:29.069912    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr" event={"ID":"06917ca2-46da-44d6-8c3c-c89f251126fb","Type":"ContainerStarted","Data":"66f7e876f25ee206535400526cb3c2671693c201e2118a48f733b440299798b2"}
Sep 14 08:16:29.099565 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:29.099538    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"]
Sep 14 08:16:29.100731 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:16:29.100704    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcc72d3c_a579_42fa_bb7e_c75c42f99954.slice/crio-414862b628352163af442d0a6318dfa1a9d07a107aa8dc294e7cd09b69c55297 WatchSource:0}: Error finding container 414862b628352163af442d0a6318dfa1a9d07a107aa8dc294e7cd09b69c55297: Status 404 returned error can't find the container with id 414862b628352163af442d0a6318dfa1a9d07a107aa8dc294e7cd09b69c55297
Sep 14 08:16:30.080162 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:30.080111    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7" event={"ID":"dcc72d3c-a579-42fa-bb7e-c75c42f99954","Type":"ContainerStarted","Data":"8eeac02a62e07eb1c247b6160c842c85d732f6b5c480f57bb6eac5e95169049f"}
Sep 14 08:16:30.080621 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:30.080181    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7" event={"ID":"dcc72d3c-a579-42fa-bb7e-c75c42f99954","Type":"ContainerStarted","Data":"414862b628352163af442d0a6318dfa1a9d07a107aa8dc294e7cd09b69c55297"}
Sep 14 08:16:30.080621 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:30.080310    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"
Sep 14 08:16:30.081873 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:30.081847    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr" event={"ID":"06917ca2-46da-44d6-8c3c-c89f251126fb","Type":"ContainerStarted","Data":"dd2996b79c024e9c1e5ba5b72f34742c27b6a5a14b91c7bb9a671048445cd1e6"}
Sep 14 08:16:30.082031 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:30.081893    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"
Sep 14 08:16:30.099432 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:30.099360    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7" podStartSLOduration=2.099341076 podStartE2EDuration="2.099341076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:16:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:16:30.095369641 +0000 UTC m=+1710.724257292" watchObservedRunningTime="2026-09-14 08:16:30.099341076 +0000 UTC m=+1710.728228712"
Sep 14 08:16:30.112525 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:30.112477    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr" podStartSLOduration=2.112461686 podStartE2EDuration="2.112461686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:16:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:16:30.109692379 +0000 UTC m=+1710.738580005" watchObservedRunningTime="2026-09-14 08:16:30.112461686 +0000 UTC m=+1710.741349316"
Sep 14 08:16:41.090427 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:41.090394    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"
Sep 14 08:16:41.090851 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:16:41.090467    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"
Sep 14 08:17:25.653841 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:17:25.653809    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jhhdc_78828de1-4078-4002-a86b-da44a9b7120f/payload-processing/0.log"
Sep 14 08:17:59.968235 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:17:59.968200    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 08:17:59.974231 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:17:59.974205    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 08:18:06.694503 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:06.694464    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7_dcc72d3c-a579-42fa-bb7e-c75c42f99954/payload-processing/0.log"
Sep 14 08:18:06.814946 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:06.814917    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jhhdc_78828de1-4078-4002-a86b-da44a9b7120f/payload-processing/0.log"
Sep 14 08:18:14.918399 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:14.918366    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"]
Sep 14 08:18:14.922096 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:14.922078    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"
Sep 14 08:18:14.924104 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:14.924079    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-925f2a\""
Sep 14 08:18:14.924242 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:14.924180    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-925f2a-dockercfg-82d95\""
Sep 14 08:18:14.931797 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:14.931771    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"]
Sep 14 08:18:14.936864 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:14.936842    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6987g\" (UniqueName: \"kubernetes.io/projected/7a6ea64d-945f-4cda-8c5e-4ec2fbd49452-kube-api-access-6987g\") pod \"payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt\" (UID: \"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"
Sep 14 08:18:14.936972 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:14.936894    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a6ea64d-945f-4cda-8c5e-4ec2fbd49452-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt\" (UID: \"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"
Sep 14 08:18:14.950767 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:14.950737    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"]
Sep 14 08:18:14.957782 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:14.957761    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"
Sep 14 08:18:14.960066 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:14.960044    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"]
Sep 14 08:18:15.038231 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.038055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22679369-1095-4568-afbe-a9e338095502-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll\" (UID: \"22679369-1095-4568-afbe-a9e338095502\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"
Sep 14 08:18:15.038231 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.038115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6987g\" (UniqueName: \"kubernetes.io/projected/7a6ea64d-945f-4cda-8c5e-4ec2fbd49452-kube-api-access-6987g\") pod \"payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt\" (UID: \"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"
Sep 14 08:18:15.038231 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.038193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a6ea64d-945f-4cda-8c5e-4ec2fbd49452-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt\" (UID: \"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"
Sep 14 08:18:15.038231 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.038221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n22d4\" (UniqueName: \"kubernetes.io/projected/22679369-1095-4568-afbe-a9e338095502-kube-api-access-n22d4\") pod \"payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll\" (UID: \"22679369-1095-4568-afbe-a9e338095502\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"
Sep 14 08:18:15.039821 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.039744    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a6ea64d-945f-4cda-8c5e-4ec2fbd49452-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt\" (UID: \"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"
Sep 14 08:18:15.045957 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.045935    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6987g\" (UniqueName: \"kubernetes.io/projected/7a6ea64d-945f-4cda-8c5e-4ec2fbd49452-kube-api-access-6987g\") pod \"payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt\" (UID: \"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"
Sep 14 08:18:15.139207 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.139169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n22d4\" (UniqueName: \"kubernetes.io/projected/22679369-1095-4568-afbe-a9e338095502-kube-api-access-n22d4\") pod \"payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll\" (UID: \"22679369-1095-4568-afbe-a9e338095502\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"
Sep 14 08:18:15.139355 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.139281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22679369-1095-4568-afbe-a9e338095502-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll\" (UID: \"22679369-1095-4568-afbe-a9e338095502\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"
Sep 14 08:18:15.140174 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.140150    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22679369-1095-4568-afbe-a9e338095502-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll\" (UID: \"22679369-1095-4568-afbe-a9e338095502\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"
Sep 14 08:18:15.146929 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.146909    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n22d4\" (UniqueName: \"kubernetes.io/projected/22679369-1095-4568-afbe-a9e338095502-kube-api-access-n22d4\") pod \"payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll\" (UID: \"22679369-1095-4568-afbe-a9e338095502\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"
Sep 14 08:18:15.232994 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.232936    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"
Sep 14 08:18:15.268452 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.268422    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"
Sep 14 08:18:15.507002 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.506972    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"]
Sep 14 08:18:15.507518 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:18:15.507495    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a6ea64d_945f_4cda_8c5e_4ec2fbd49452.slice/crio-d3c6f1db59a4914517f268c3b0e23495527d4f55d00c7b59c36ca10d711a3d41 WatchSource:0}: Error finding container d3c6f1db59a4914517f268c3b0e23495527d4f55d00c7b59c36ca10d711a3d41: Status 404 returned error can't find the container with id d3c6f1db59a4914517f268c3b0e23495527d4f55d00c7b59c36ca10d711a3d41
Sep 14 08:18:15.554328 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.554305    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"]
Sep 14 08:18:15.565714 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:18:15.565689    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22679369_1095_4568_afbe_a9e338095502.slice/crio-e2c130cb17bd21bfedb628574ffa9225c3e7de8cad8c1beb1dff87cfebee57d4 WatchSource:0}: Error finding container e2c130cb17bd21bfedb628574ffa9225c3e7de8cad8c1beb1dff87cfebee57d4: Status 404 returned error can't find the container with id e2c130cb17bd21bfedb628574ffa9225c3e7de8cad8c1beb1dff87cfebee57d4
Sep 14 08:18:15.709926 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.709876    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll" event={"ID":"22679369-1095-4568-afbe-a9e338095502","Type":"ContainerStarted","Data":"7a969c43d6f55f9d76ebfb54bdf5d08c92417ef9b39bf22cd1ab5eff22d1f0a1"}
Sep 14 08:18:15.710210 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.710013    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll" event={"ID":"22679369-1095-4568-afbe-a9e338095502","Type":"ContainerStarted","Data":"e2c130cb17bd21bfedb628574ffa9225c3e7de8cad8c1beb1dff87cfebee57d4"}
Sep 14 08:18:15.710210 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.710070    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"
Sep 14 08:18:15.711784 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.711761    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt" event={"ID":"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452","Type":"ContainerStarted","Data":"edc31d7e07221a64c007c02f01d66eb3cbc3d5b22b35da55e73c94a39652fd2a"}
Sep 14 08:18:15.711890 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.711800    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt" event={"ID":"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452","Type":"ContainerStarted","Data":"d3c6f1db59a4914517f268c3b0e23495527d4f55d00c7b59c36ca10d711a3d41"}
Sep 14 08:18:15.711890 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.711865    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"
Sep 14 08:18:15.730054 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.730004    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll" podStartSLOduration=1.7299863530000001 podStartE2EDuration="1.729986353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:18:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:18:15.725272968 +0000 UTC m=+1816.354160597" watchObservedRunningTime="2026-09-14 08:18:15.729986353 +0000 UTC m=+1816.358873983"
Sep 14 08:18:15.742603 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:15.742551    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt" podStartSLOduration=1.742533758 podStartE2EDuration="1.742533758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:18:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:18:15.739846454 +0000 UTC m=+1816.368734103" watchObservedRunningTime="2026-09-14 08:18:15.742533758 +0000 UTC m=+1816.371421386"
Sep 14 08:18:26.720546 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:26.720510    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"
Sep 14 08:18:26.720991 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:26.720598    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"
Sep 14 08:18:41.660462 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:41.660426    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf"]
Sep 14 08:18:41.664115 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:41.664089    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf"
Sep 14 08:18:41.666720 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:41.666697    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:18:41.672163 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:41.672135    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf"]
Sep 14 08:18:41.769806 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:41.769772    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9c440f6-e6bb-4468-83fe-ca494e0f7878-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf\" (UID: \"b9c440f6-e6bb-4468-83fe-ca494e0f7878\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf"
Sep 14 08:18:41.870405 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:41.870367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9c440f6-e6bb-4468-83fe-ca494e0f7878-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf\" (UID: \"b9c440f6-e6bb-4468-83fe-ca494e0f7878\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf"
Sep 14 08:18:41.871152 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:41.871123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9c440f6-e6bb-4468-83fe-ca494e0f7878-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf\" (UID: \"b9c440f6-e6bb-4468-83fe-ca494e0f7878\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf"
Sep 14 08:18:41.979290 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:41.979224    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf"
Sep 14 08:18:42.250781 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:42.250750    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf"]
Sep 14 08:18:42.251953 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:18:42.251927    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9c440f6_e6bb_4468_83fe_ca494e0f7878.slice/crio-3827bc52f581d4c5d4960ed69fcc76b8c1f1a54701362fb81b7ba4563af17279 WatchSource:0}: Error finding container 3827bc52f581d4c5d4960ed69fcc76b8c1f1a54701362fb81b7ba4563af17279: Status 404 returned error can't find the container with id 3827bc52f581d4c5d4960ed69fcc76b8c1f1a54701362fb81b7ba4563af17279
Sep 14 08:18:42.876928 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:42.876894    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf" event={"ID":"b9c440f6-e6bb-4468-83fe-ca494e0f7878","Type":"ContainerStarted","Data":"77774c99959cd1d15b69127a190ee7f493cf63063359ca6d04f2a23c4a61cac3"}
Sep 14 08:18:42.877347 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:42.876949    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf" event={"ID":"b9c440f6-e6bb-4468-83fe-ca494e0f7878","Type":"ContainerStarted","Data":"3827bc52f581d4c5d4960ed69fcc76b8c1f1a54701362fb81b7ba4563af17279"}
Sep 14 08:18:42.893722 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:42.893672    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf" podStartSLOduration=1.893631277 podStartE2EDuration="1.893631277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:18:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:18:42.892957792 +0000 UTC m=+1843.521845440" watchObservedRunningTime="2026-09-14 08:18:42.893631277 +0000 UTC m=+1843.522518908"
Sep 14 08:18:43.884285 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:43.884255    2563 generic.go:356] "Generic (PLEG): container finished" podID="b9c440f6-e6bb-4468-83fe-ca494e0f7878" containerID="77774c99959cd1d15b69127a190ee7f493cf63063359ca6d04f2a23c4a61cac3" exitCode=0
Sep 14 08:18:43.886291 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:43.886264    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf" event={"ID":"b9c440f6-e6bb-4468-83fe-ca494e0f7878","Type":"ContainerDied","Data":"77774c99959cd1d15b69127a190ee7f493cf63063359ca6d04f2a23c4a61cac3"}
Sep 14 08:18:45.031430 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:45.031408    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf"
Sep 14 08:18:45.099340 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:45.099308    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9c440f6-e6bb-4468-83fe-ca494e0f7878-maas-api-service-ca\") pod \"b9c440f6-e6bb-4468-83fe-ca494e0f7878\" (UID: \"b9c440f6-e6bb-4468-83fe-ca494e0f7878\") "
Sep 14 08:18:45.099678 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:45.099629    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9c440f6-e6bb-4468-83fe-ca494e0f7878-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b9c440f6-e6bb-4468-83fe-ca494e0f7878" (UID: "b9c440f6-e6bb-4468-83fe-ca494e0f7878"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:18:45.200674 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:45.200576    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9c440f6-e6bb-4468-83fe-ca494e0f7878-maas-api-service-ca\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:18:45.896776 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:45.896749    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf"
Sep 14 08:18:45.896885 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:45.896780    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf" event={"ID":"b9c440f6-e6bb-4468-83fe-ca494e0f7878","Type":"ContainerDied","Data":"3827bc52f581d4c5d4960ed69fcc76b8c1f1a54701362fb81b7ba4563af17279"}
Sep 14 08:18:45.896885 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:45.896818    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3827bc52f581d4c5d4960ed69fcc76b8c1f1a54701362fb81b7ba4563af17279"
Sep 14 08:18:52.963444 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:52.963400    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"]
Sep 14 08:18:52.963928 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:52.963708    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll" podUID="22679369-1095-4568-afbe-a9e338095502" containerName="payload-processing" containerID="cri-o://7a969c43d6f55f9d76ebfb54bdf5d08c92417ef9b39bf22cd1ab5eff22d1f0a1" gracePeriod=30
Sep 14 08:18:52.967028 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:52.967002    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"]
Sep 14 08:18:52.967656 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:52.967563    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt" podUID="7a6ea64d-945f-4cda-8c5e-4ec2fbd49452" containerName="payload-pre-processing" containerID="cri-o://edc31d7e07221a64c007c02f01d66eb3cbc3d5b22b35da55e73c94a39652fd2a" gracePeriod=30
Sep 14 08:18:53.159124 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.159092    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf"]
Sep 14 08:18:53.162105 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.162078    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-925f2a-jwtkf"]
Sep 14 08:18:53.240924 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.240895    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"
Sep 14 08:18:53.255535 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.255511    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"
Sep 14 08:18:53.371000 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.370972    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n22d4\" (UniqueName: \"kubernetes.io/projected/22679369-1095-4568-afbe-a9e338095502-kube-api-access-n22d4\") pod \"22679369-1095-4568-afbe-a9e338095502\" (UID: \"22679369-1095-4568-afbe-a9e338095502\") "
Sep 14 08:18:53.371162 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.371022    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a6ea64d-945f-4cda-8c5e-4ec2fbd49452-plugins-config-volume\") pod \"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452\" (UID: \"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452\") "
Sep 14 08:18:53.371239 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.371221    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6987g\" (UniqueName: \"kubernetes.io/projected/7a6ea64d-945f-4cda-8c5e-4ec2fbd49452-kube-api-access-6987g\") pod \"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452\" (UID: \"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452\") "
Sep 14 08:18:53.371297 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.371272    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22679369-1095-4568-afbe-a9e338095502-plugins-config-volume\") pod \"22679369-1095-4568-afbe-a9e338095502\" (UID: \"22679369-1095-4568-afbe-a9e338095502\") "
Sep 14 08:18:53.371389 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.371364    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a6ea64d-945f-4cda-8c5e-4ec2fbd49452-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a6ea64d-945f-4cda-8c5e-4ec2fbd49452" (UID: "7a6ea64d-945f-4cda-8c5e-4ec2fbd49452"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:18:53.371689 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.371630    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a6ea64d-945f-4cda-8c5e-4ec2fbd49452-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:18:53.371822 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.371704    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22679369-1095-4568-afbe-a9e338095502-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "22679369-1095-4568-afbe-a9e338095502" (UID: "22679369-1095-4568-afbe-a9e338095502"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:18:53.373233 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.373206    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22679369-1095-4568-afbe-a9e338095502-kube-api-access-n22d4" (OuterVolumeSpecName: "kube-api-access-n22d4") pod "22679369-1095-4568-afbe-a9e338095502" (UID: "22679369-1095-4568-afbe-a9e338095502"). InnerVolumeSpecName "kube-api-access-n22d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:18:53.373422 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.373399    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a6ea64d-945f-4cda-8c5e-4ec2fbd49452-kube-api-access-6987g" (OuterVolumeSpecName: "kube-api-access-6987g") pod "7a6ea64d-945f-4cda-8c5e-4ec2fbd49452" (UID: "7a6ea64d-945f-4cda-8c5e-4ec2fbd49452"). InnerVolumeSpecName "kube-api-access-6987g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:18:53.472357 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.472281    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6987g\" (UniqueName: \"kubernetes.io/projected/7a6ea64d-945f-4cda-8c5e-4ec2fbd49452-kube-api-access-6987g\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:18:53.472357 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.472313    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22679369-1095-4568-afbe-a9e338095502-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:18:53.472357 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.472323    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n22d4\" (UniqueName: \"kubernetes.io/projected/22679369-1095-4568-afbe-a9e338095502-kube-api-access-n22d4\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:18:53.886605 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.886568    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9c440f6-e6bb-4468-83fe-ca494e0f7878" path="/var/lib/kubelet/pods/b9c440f6-e6bb-4468-83fe-ca494e0f7878/volumes"
Sep 14 08:18:53.945518 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.945489    2563 generic.go:356] "Generic (PLEG): container finished" podID="7a6ea64d-945f-4cda-8c5e-4ec2fbd49452" containerID="edc31d7e07221a64c007c02f01d66eb3cbc3d5b22b35da55e73c94a39652fd2a" exitCode=0
Sep 14 08:18:53.945688 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.945537    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"
Sep 14 08:18:53.945688 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.945585    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt" event={"ID":"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452","Type":"ContainerDied","Data":"edc31d7e07221a64c007c02f01d66eb3cbc3d5b22b35da55e73c94a39652fd2a"}
Sep 14 08:18:53.945688 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.945664    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt" event={"ID":"7a6ea64d-945f-4cda-8c5e-4ec2fbd49452","Type":"ContainerDied","Data":"d3c6f1db59a4914517f268c3b0e23495527d4f55d00c7b59c36ca10d711a3d41"}
Sep 14 08:18:53.945688 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.945680    2563 scope.go:117] "RemoveContainer" containerID="edc31d7e07221a64c007c02f01d66eb3cbc3d5b22b35da55e73c94a39652fd2a"
Sep 14 08:18:53.947661 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.947571    2563 generic.go:356] "Generic (PLEG): container finished" podID="22679369-1095-4568-afbe-a9e338095502" containerID="7a969c43d6f55f9d76ebfb54bdf5d08c92417ef9b39bf22cd1ab5eff22d1f0a1" exitCode=0
Sep 14 08:18:53.947661 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.947610    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"
Sep 14 08:18:53.947661 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.947630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll" event={"ID":"22679369-1095-4568-afbe-a9e338095502","Type":"ContainerDied","Data":"7a969c43d6f55f9d76ebfb54bdf5d08c92417ef9b39bf22cd1ab5eff22d1f0a1"}
Sep 14 08:18:53.947869 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.947691    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll" event={"ID":"22679369-1095-4568-afbe-a9e338095502","Type":"ContainerDied","Data":"e2c130cb17bd21bfedb628574ffa9225c3e7de8cad8c1beb1dff87cfebee57d4"}
Sep 14 08:18:53.955657 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.955626    2563 scope.go:117] "RemoveContainer" containerID="edc31d7e07221a64c007c02f01d66eb3cbc3d5b22b35da55e73c94a39652fd2a"
Sep 14 08:18:53.955939 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:18:53.955921    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edc31d7e07221a64c007c02f01d66eb3cbc3d5b22b35da55e73c94a39652fd2a\": container with ID starting with edc31d7e07221a64c007c02f01d66eb3cbc3d5b22b35da55e73c94a39652fd2a not found: ID does not exist" containerID="edc31d7e07221a64c007c02f01d66eb3cbc3d5b22b35da55e73c94a39652fd2a"
Sep 14 08:18:53.956019 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.955947    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edc31d7e07221a64c007c02f01d66eb3cbc3d5b22b35da55e73c94a39652fd2a"} err="failed to get container status \"edc31d7e07221a64c007c02f01d66eb3cbc3d5b22b35da55e73c94a39652fd2a\": rpc error: code = NotFound desc = could not find container \"edc31d7e07221a64c007c02f01d66eb3cbc3d5b22b35da55e73c94a39652fd2a\": container with ID starting with edc31d7e07221a64c007c02f01d66eb3cbc3d5b22b35da55e73c94a39652fd2a not found: ID does not exist"
Sep 14 08:18:53.956019 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.955963    2563 scope.go:117] "RemoveContainer" containerID="7a969c43d6f55f9d76ebfb54bdf5d08c92417ef9b39bf22cd1ab5eff22d1f0a1"
Sep 14 08:18:53.963430 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.963407    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"]
Sep 14 08:18:53.964979 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.964802    2563 scope.go:117] "RemoveContainer" containerID="7a969c43d6f55f9d76ebfb54bdf5d08c92417ef9b39bf22cd1ab5eff22d1f0a1"
Sep 14 08:18:53.965310 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:18:53.965187    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a969c43d6f55f9d76ebfb54bdf5d08c92417ef9b39bf22cd1ab5eff22d1f0a1\": container with ID starting with 7a969c43d6f55f9d76ebfb54bdf5d08c92417ef9b39bf22cd1ab5eff22d1f0a1 not found: ID does not exist" containerID="7a969c43d6f55f9d76ebfb54bdf5d08c92417ef9b39bf22cd1ab5eff22d1f0a1"
Sep 14 08:18:53.965310 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.965217    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a969c43d6f55f9d76ebfb54bdf5d08c92417ef9b39bf22cd1ab5eff22d1f0a1"} err="failed to get container status \"7a969c43d6f55f9d76ebfb54bdf5d08c92417ef9b39bf22cd1ab5eff22d1f0a1\": rpc error: code = NotFound desc = could not find container \"7a969c43d6f55f9d76ebfb54bdf5d08c92417ef9b39bf22cd1ab5eff22d1f0a1\": container with ID starting with 7a969c43d6f55f9d76ebfb54bdf5d08c92417ef9b39bf22cd1ab5eff22d1f0a1 not found: ID does not exist"
Sep 14 08:18:53.968230 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.968212    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-925f2a-57f54b788b-622rt"]
Sep 14 08:18:53.977710 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.977689    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"]
Sep 14 08:18:53.982424 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:53.982396    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-925f2a-c6b9bd5-fbhll"]
Sep 14 08:18:55.886169 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:55.886126    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22679369-1095-4568-afbe-a9e338095502" path="/var/lib/kubelet/pods/22679369-1095-4568-afbe-a9e338095502/volumes"
Sep 14 08:18:55.886707 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:18:55.886687    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a6ea64d-945f-4cda-8c5e-4ec2fbd49452" path="/var/lib/kubelet/pods/7a6ea64d-945f-4cda-8c5e-4ec2fbd49452/volumes"
Sep 14 08:19:00.224654 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:00.224619    2563 scope.go:117] "RemoveContainer" containerID="d0d5669951315f18a8f624802b96e42f7c3f067fc9a505f1b76d147c708e6c36"
Sep 14 08:19:00.233002 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:00.232982    2563 scope.go:117] "RemoveContainer" containerID="62aef19086ddaee0552e884f638d22363720c0e731f00c16b8363599e8c2ba1f"
Sep 14 08:19:00.241709 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:00.241685    2563 scope.go:117] "RemoveContainer" containerID="95a765229243964d2753729479bf07fc13cdc5747cd52c45517519e0758178b8"
Sep 14 08:19:03.417726 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.417692    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58"]
Sep 14 08:19:03.418134 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.418120    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a6ea64d-945f-4cda-8c5e-4ec2fbd49452" containerName="payload-pre-processing"
Sep 14 08:19:03.418178 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.418135    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a6ea64d-945f-4cda-8c5e-4ec2fbd49452" containerName="payload-pre-processing"
Sep 14 08:19:03.418178 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.418151    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9c440f6-e6bb-4468-83fe-ca494e0f7878" containerName="revoke-keys"
Sep 14 08:19:03.418178 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.418156    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9c440f6-e6bb-4468-83fe-ca494e0f7878" containerName="revoke-keys"
Sep 14 08:19:03.418178 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.418167    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22679369-1095-4568-afbe-a9e338095502" containerName="payload-processing"
Sep 14 08:19:03.418178 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.418173    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="22679369-1095-4568-afbe-a9e338095502" containerName="payload-processing"
Sep 14 08:19:03.418327 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.418248    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="22679369-1095-4568-afbe-a9e338095502" containerName="payload-processing"
Sep 14 08:19:03.418327 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.418255    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9c440f6-e6bb-4468-83fe-ca494e0f7878" containerName="revoke-keys"
Sep 14 08:19:03.418327 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.418260    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a6ea64d-945f-4cda-8c5e-4ec2fbd49452" containerName="payload-pre-processing"
Sep 14 08:19:03.421332 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.421312    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58"
Sep 14 08:19:03.423477 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.423457    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:19:03.428121 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.428099    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58"]
Sep 14 08:19:03.569699 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.569663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57bc36ca-b87f-4195-8d35-29f434d7f1e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58\" (UID: \"57bc36ca-b87f-4195-8d35-29f434d7f1e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58"
Sep 14 08:19:03.671117 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.671010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57bc36ca-b87f-4195-8d35-29f434d7f1e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58\" (UID: \"57bc36ca-b87f-4195-8d35-29f434d7f1e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58"
Sep 14 08:19:03.671675 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.671655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57bc36ca-b87f-4195-8d35-29f434d7f1e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58\" (UID: \"57bc36ca-b87f-4195-8d35-29f434d7f1e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58"
Sep 14 08:19:03.732361 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:03.732310    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58"
Sep 14 08:19:04.005827 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:04.005796    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58"]
Sep 14 08:19:04.006958 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:19:04.006932    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57bc36ca_b87f_4195_8d35_29f434d7f1e1.slice/crio-9dcd5c7081a9052b9c02093711f636a60aa516500f4fd1ba2887626dcc20ba09 WatchSource:0}: Error finding container 9dcd5c7081a9052b9c02093711f636a60aa516500f4fd1ba2887626dcc20ba09: Status 404 returned error can't find the container with id 9dcd5c7081a9052b9c02093711f636a60aa516500f4fd1ba2887626dcc20ba09
Sep 14 08:19:05.018086 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:05.018005    2563 generic.go:356] "Generic (PLEG): container finished" podID="57bc36ca-b87f-4195-8d35-29f434d7f1e1" containerID="e4853d69e7c802c3ff5c00f40817e924605b931b95adebd0f988181951d91aca" exitCode=0
Sep 14 08:19:05.018086 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:05.018056    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58" event={"ID":"57bc36ca-b87f-4195-8d35-29f434d7f1e1","Type":"ContainerDied","Data":"e4853d69e7c802c3ff5c00f40817e924605b931b95adebd0f988181951d91aca"}
Sep 14 08:19:05.018481 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:05.018113    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58" event={"ID":"57bc36ca-b87f-4195-8d35-29f434d7f1e1","Type":"ContainerStarted","Data":"9dcd5c7081a9052b9c02093711f636a60aa516500f4fd1ba2887626dcc20ba09"}
Sep 14 08:19:06.156991 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:06.156965    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58"
Sep 14 08:19:06.294860 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:06.294772    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57bc36ca-b87f-4195-8d35-29f434d7f1e1-maas-api-service-ca\") pod \"57bc36ca-b87f-4195-8d35-29f434d7f1e1\" (UID: \"57bc36ca-b87f-4195-8d35-29f434d7f1e1\") "
Sep 14 08:19:06.295141 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:06.295114    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57bc36ca-b87f-4195-8d35-29f434d7f1e1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "57bc36ca-b87f-4195-8d35-29f434d7f1e1" (UID: "57bc36ca-b87f-4195-8d35-29f434d7f1e1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:19:06.396346 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:06.396310    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/57bc36ca-b87f-4195-8d35-29f434d7f1e1-maas-api-service-ca\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:19:07.030110 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:07.030081    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58"
Sep 14 08:19:07.030110 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:07.030107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58" event={"ID":"57bc36ca-b87f-4195-8d35-29f434d7f1e1","Type":"ContainerDied","Data":"9dcd5c7081a9052b9c02093711f636a60aa516500f4fd1ba2887626dcc20ba09"}
Sep 14 08:19:07.030337 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:07.030147    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9dcd5c7081a9052b9c02093711f636a60aa516500f4fd1ba2887626dcc20ba09"
Sep 14 08:19:18.974515 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:18.974479    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"]
Sep 14 08:19:18.975139 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:18.974726    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7" podUID="dcc72d3c-a579-42fa-bb7e-c75c42f99954" containerName="payload-processing" containerID="cri-o://8eeac02a62e07eb1c247b6160c842c85d732f6b5c480f57bb6eac5e95169049f" gracePeriod=30
Sep 14 08:19:18.983943 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:18.983913    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"]
Sep 14 08:19:18.984150 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:18.984124    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr" podUID="06917ca2-46da-44d6-8c3c-c89f251126fb" containerName="payload-pre-processing" containerID="cri-o://dd2996b79c024e9c1e5ba5b72f34742c27b6a5a14b91c7bb9a671048445cd1e6" gracePeriod=30
Sep 14 08:19:19.103497 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.103430    2563 generic.go:356] "Generic (PLEG): container finished" podID="06917ca2-46da-44d6-8c3c-c89f251126fb" containerID="dd2996b79c024e9c1e5ba5b72f34742c27b6a5a14b91c7bb9a671048445cd1e6" exitCode=0
Sep 14 08:19:19.103602 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.103507    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr" event={"ID":"06917ca2-46da-44d6-8c3c-c89f251126fb","Type":"ContainerDied","Data":"dd2996b79c024e9c1e5ba5b72f34742c27b6a5a14b91c7bb9a671048445cd1e6"}
Sep 14 08:19:19.105718 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.105689    2563 generic.go:356] "Generic (PLEG): container finished" podID="dcc72d3c-a579-42fa-bb7e-c75c42f99954" containerID="8eeac02a62e07eb1c247b6160c842c85d732f6b5c480f57bb6eac5e95169049f" exitCode=0
Sep 14 08:19:19.105821 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.105741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7" event={"ID":"dcc72d3c-a579-42fa-bb7e-c75c42f99954","Type":"ContainerDied","Data":"8eeac02a62e07eb1c247b6160c842c85d732f6b5c480f57bb6eac5e95169049f"}
Sep 14 08:19:19.150692 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.148850    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58"]
Sep 14 08:19:19.152943 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.152757    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-7f1623-rsd58"]
Sep 14 08:19:19.273304 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.273220    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"
Sep 14 08:19:19.276279 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.276259    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"
Sep 14 08:19:19.317359 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.317330    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06917ca2-46da-44d6-8c3c-c89f251126fb-plugins-config-volume\") pod \"06917ca2-46da-44d6-8c3c-c89f251126fb\" (UID: \"06917ca2-46da-44d6-8c3c-c89f251126fb\") "
Sep 14 08:19:19.317543 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.317470    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9s88\" (UniqueName: \"kubernetes.io/projected/06917ca2-46da-44d6-8c3c-c89f251126fb-kube-api-access-h9s88\") pod \"06917ca2-46da-44d6-8c3c-c89f251126fb\" (UID: \"06917ca2-46da-44d6-8c3c-c89f251126fb\") "
Sep 14 08:19:19.317543 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.317488    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49phx\" (UniqueName: \"kubernetes.io/projected/dcc72d3c-a579-42fa-bb7e-c75c42f99954-kube-api-access-49phx\") pod \"dcc72d3c-a579-42fa-bb7e-c75c42f99954\" (UID: \"dcc72d3c-a579-42fa-bb7e-c75c42f99954\") "
Sep 14 08:19:19.317543 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.317513    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcc72d3c-a579-42fa-bb7e-c75c42f99954-plugins-config-volume\") pod \"dcc72d3c-a579-42fa-bb7e-c75c42f99954\" (UID: \"dcc72d3c-a579-42fa-bb7e-c75c42f99954\") "
Sep 14 08:19:19.317758 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.317724    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06917ca2-46da-44d6-8c3c-c89f251126fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "06917ca2-46da-44d6-8c3c-c89f251126fb" (UID: "06917ca2-46da-44d6-8c3c-c89f251126fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:19:19.317840 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.317819    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06917ca2-46da-44d6-8c3c-c89f251126fb-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:19:19.318076 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.318045    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcc72d3c-a579-42fa-bb7e-c75c42f99954-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dcc72d3c-a579-42fa-bb7e-c75c42f99954" (UID: "dcc72d3c-a579-42fa-bb7e-c75c42f99954"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:19:19.319987 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.319931    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06917ca2-46da-44d6-8c3c-c89f251126fb-kube-api-access-h9s88" (OuterVolumeSpecName: "kube-api-access-h9s88") pod "06917ca2-46da-44d6-8c3c-c89f251126fb" (UID: "06917ca2-46da-44d6-8c3c-c89f251126fb"). InnerVolumeSpecName "kube-api-access-h9s88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:19:19.319987 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.319943    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcc72d3c-a579-42fa-bb7e-c75c42f99954-kube-api-access-49phx" (OuterVolumeSpecName: "kube-api-access-49phx") pod "dcc72d3c-a579-42fa-bb7e-c75c42f99954" (UID: "dcc72d3c-a579-42fa-bb7e-c75c42f99954"). InnerVolumeSpecName "kube-api-access-49phx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:19:19.418687 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.418656    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9s88\" (UniqueName: \"kubernetes.io/projected/06917ca2-46da-44d6-8c3c-c89f251126fb-kube-api-access-h9s88\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:19:19.418687 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.418683    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49phx\" (UniqueName: \"kubernetes.io/projected/dcc72d3c-a579-42fa-bb7e-c75c42f99954-kube-api-access-49phx\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:19:19.418687 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.418693    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dcc72d3c-a579-42fa-bb7e-c75c42f99954-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:19:19.887226 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:19.887189    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57bc36ca-b87f-4195-8d35-29f434d7f1e1" path="/var/lib/kubelet/pods/57bc36ca-b87f-4195-8d35-29f434d7f1e1/volumes"
Sep 14 08:19:20.112857 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:20.112826    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"
Sep 14 08:19:20.113289 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:20.112867    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7" event={"ID":"dcc72d3c-a579-42fa-bb7e-c75c42f99954","Type":"ContainerDied","Data":"414862b628352163af442d0a6318dfa1a9d07a107aa8dc294e7cd09b69c55297"}
Sep 14 08:19:20.113289 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:20.112942    2563 scope.go:117] "RemoveContainer" containerID="8eeac02a62e07eb1c247b6160c842c85d732f6b5c480f57bb6eac5e95169049f"
Sep 14 08:19:20.114484 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:20.114457    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"
Sep 14 08:19:20.114484 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:20.114473    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr" event={"ID":"06917ca2-46da-44d6-8c3c-c89f251126fb","Type":"ContainerDied","Data":"66f7e876f25ee206535400526cb3c2671693c201e2118a48f733b440299798b2"}
Sep 14 08:19:20.122103 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:20.122089    2563 scope.go:117] "RemoveContainer" containerID="dd2996b79c024e9c1e5ba5b72f34742c27b6a5a14b91c7bb9a671048445cd1e6"
Sep 14 08:19:20.133341 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:20.133322    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"]
Sep 14 08:19:20.137090 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:20.137069    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7f1623-6f8bb849c5-gzml7"]
Sep 14 08:19:20.145929 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:20.145878    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"]
Sep 14 08:19:20.149301 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:20.149281    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7f1623-ccbdffd6b-xzkvr"]
Sep 14 08:19:21.886323 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:21.886293    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06917ca2-46da-44d6-8c3c-c89f251126fb" path="/var/lib/kubelet/pods/06917ca2-46da-44d6-8c3c-c89f251126fb/volumes"
Sep 14 08:19:21.886723 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:19:21.886702    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcc72d3c-a579-42fa-bb7e-c75c42f99954" path="/var/lib/kubelet/pods/dcc72d3c-a579-42fa-bb7e-c75c42f99954/volumes"
Sep 14 08:20:13.374672 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.372911    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw"]
Sep 14 08:20:13.374672 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.373918    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57bc36ca-b87f-4195-8d35-29f434d7f1e1" containerName="revoke-keys"
Sep 14 08:20:13.374672 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.373942    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="57bc36ca-b87f-4195-8d35-29f434d7f1e1" containerName="revoke-keys"
Sep 14 08:20:13.374672 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.373980    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcc72d3c-a579-42fa-bb7e-c75c42f99954" containerName="payload-processing"
Sep 14 08:20:13.374672 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.373989    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcc72d3c-a579-42fa-bb7e-c75c42f99954" containerName="payload-processing"
Sep 14 08:20:13.374672 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.374010    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06917ca2-46da-44d6-8c3c-c89f251126fb" containerName="payload-pre-processing"
Sep 14 08:20:13.374672 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.374019    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="06917ca2-46da-44d6-8c3c-c89f251126fb" containerName="payload-pre-processing"
Sep 14 08:20:13.374672 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.374202    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="57bc36ca-b87f-4195-8d35-29f434d7f1e1" containerName="revoke-keys"
Sep 14 08:20:13.374672 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.374223    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcc72d3c-a579-42fa-bb7e-c75c42f99954" containerName="payload-processing"
Sep 14 08:20:13.374672 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.374243    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="06917ca2-46da-44d6-8c3c-c89f251126fb" containerName="payload-pre-processing"
Sep 14 08:20:13.380198 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.380171    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw"
Sep 14 08:20:13.382945 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.382889    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:20:13.384417 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.384359    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw"]
Sep 14 08:20:13.410751 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.410726    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw\" (UID: \"8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw"
Sep 14 08:20:13.511708 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.511665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw\" (UID: \"8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw"
Sep 14 08:20:13.512306 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.512284    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw\" (UID: \"8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw"
Sep 14 08:20:13.695814 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.695723    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw"
Sep 14 08:20:13.966857 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:13.966833    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw"]
Sep 14 08:20:13.967984 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:20:13.967954    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d3f3528_8780_4ea3_9dd4_1cc72ea4e1f8.slice/crio-3496472e55dac66d6b98f10c29829373f38617db55fb873252afd757b9cbb3ff WatchSource:0}: Error finding container 3496472e55dac66d6b98f10c29829373f38617db55fb873252afd757b9cbb3ff: Status 404 returned error can't find the container with id 3496472e55dac66d6b98f10c29829373f38617db55fb873252afd757b9cbb3ff
Sep 14 08:20:14.425595 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:14.425556    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw" event={"ID":"8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8","Type":"ContainerStarted","Data":"c9ecda32d63227b0765c0a40a22b07ca08ffbb687d0efe3985af2e3c52cbaa41"}
Sep 14 08:20:14.426013 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:14.425614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw" event={"ID":"8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8","Type":"ContainerStarted","Data":"3496472e55dac66d6b98f10c29829373f38617db55fb873252afd757b9cbb3ff"}
Sep 14 08:20:14.438591 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:14.438534    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw" podStartSLOduration=1.438496279 podStartE2EDuration="1.438496279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:20:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:20:14.437953536 +0000 UTC m=+1935.066841183" watchObservedRunningTime="2026-09-14 08:20:14.438496279 +0000 UTC m=+1935.067383952"
Sep 14 08:20:15.432400 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:15.432366    2563 generic.go:356] "Generic (PLEG): container finished" podID="8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8" containerID="c9ecda32d63227b0765c0a40a22b07ca08ffbb687d0efe3985af2e3c52cbaa41" exitCode=0
Sep 14 08:20:15.432850 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:15.432452    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw" event={"ID":"8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8","Type":"ContainerDied","Data":"c9ecda32d63227b0765c0a40a22b07ca08ffbb687d0efe3985af2e3c52cbaa41"}
Sep 14 08:20:16.572093 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:16.572068    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw"
Sep 14 08:20:16.637661 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:16.637607    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8-maas-api-service-ca\") pod \"8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8\" (UID: \"8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8\") "
Sep 14 08:20:16.637921 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:16.637898    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8" (UID: "8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:16.739182 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:16.739112    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8-maas-api-service-ca\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:20:17.445193 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:17.445156    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw"
Sep 14 08:20:17.445359 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:17.445199    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw" event={"ID":"8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8","Type":"ContainerDied","Data":"3496472e55dac66d6b98f10c29829373f38617db55fb873252afd757b9cbb3ff"}
Sep 14 08:20:17.445359 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:17.445260    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3496472e55dac66d6b98f10c29829373f38617db55fb873252afd757b9cbb3ff"
Sep 14 08:20:24.098003 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:24.097967    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw"]
Sep 14 08:20:24.100445 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:24.100415    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-56708c-c7pfw"]
Sep 14 08:20:25.886605 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:25.886572    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8" path="/var/lib/kubelet/pods/8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8/volumes"
Sep 14 08:20:41.478223 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.478187    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"]
Sep 14 08:20:41.478756 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.478410    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d" podUID="1e91a6a2-9aaa-47b8-9960-b417ba260f7a" containerName="payload-processing" containerID="cri-o://38676bf04b758469e04fb9a58ad4bc6ba9e81a4a8e7d13afd7130a96d74d5c92" gracePeriod=30
Sep 14 08:20:41.489055 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.489020    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"]
Sep 14 08:20:41.490925 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.490894    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld" podUID="e2c5c0e1-e434-493e-be41-f8e966a03648" containerName="payload-pre-processing" containerID="cri-o://99278a18a0bd895d9cea225f2f9353da8c09695be15d58373e7f5f485d7f30a8" gracePeriod=30
Sep 14 08:20:41.602858 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.602832    2563 generic.go:356] "Generic (PLEG): container finished" podID="1e91a6a2-9aaa-47b8-9960-b417ba260f7a" containerID="38676bf04b758469e04fb9a58ad4bc6ba9e81a4a8e7d13afd7130a96d74d5c92" exitCode=0
Sep 14 08:20:41.602993 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.602930    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d" event={"ID":"1e91a6a2-9aaa-47b8-9960-b417ba260f7a","Type":"ContainerDied","Data":"38676bf04b758469e04fb9a58ad4bc6ba9e81a4a8e7d13afd7130a96d74d5c92"}
Sep 14 08:20:41.760099 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.760070    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"
Sep 14 08:20:41.786799 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.786770    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"
Sep 14 08:20:41.869170 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.869139    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e91a6a2-9aaa-47b8-9960-b417ba260f7a-plugins-config-volume\") pod \"1e91a6a2-9aaa-47b8-9960-b417ba260f7a\" (UID: \"1e91a6a2-9aaa-47b8-9960-b417ba260f7a\") "
Sep 14 08:20:41.869343 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.869229    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z9g8\" (UniqueName: \"kubernetes.io/projected/1e91a6a2-9aaa-47b8-9960-b417ba260f7a-kube-api-access-7z9g8\") pod \"1e91a6a2-9aaa-47b8-9960-b417ba260f7a\" (UID: \"1e91a6a2-9aaa-47b8-9960-b417ba260f7a\") "
Sep 14 08:20:41.869573 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.869555    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e91a6a2-9aaa-47b8-9960-b417ba260f7a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1e91a6a2-9aaa-47b8-9960-b417ba260f7a" (UID: "1e91a6a2-9aaa-47b8-9960-b417ba260f7a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:41.871305 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.871283    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e91a6a2-9aaa-47b8-9960-b417ba260f7a-kube-api-access-7z9g8" (OuterVolumeSpecName: "kube-api-access-7z9g8") pod "1e91a6a2-9aaa-47b8-9960-b417ba260f7a" (UID: "1e91a6a2-9aaa-47b8-9960-b417ba260f7a"). InnerVolumeSpecName "kube-api-access-7z9g8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:41.970025 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.969992    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2252k\" (UniqueName: \"kubernetes.io/projected/e2c5c0e1-e434-493e-be41-f8e966a03648-kube-api-access-2252k\") pod \"e2c5c0e1-e434-493e-be41-f8e966a03648\" (UID: \"e2c5c0e1-e434-493e-be41-f8e966a03648\") "
Sep 14 08:20:41.970218 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.970054    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2c5c0e1-e434-493e-be41-f8e966a03648-plugins-config-volume\") pod \"e2c5c0e1-e434-493e-be41-f8e966a03648\" (UID: \"e2c5c0e1-e434-493e-be41-f8e966a03648\") "
Sep 14 08:20:41.970280 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.970269    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e91a6a2-9aaa-47b8-9960-b417ba260f7a-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:20:41.970337 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.970285    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z9g8\" (UniqueName: \"kubernetes.io/projected/1e91a6a2-9aaa-47b8-9960-b417ba260f7a-kube-api-access-7z9g8\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:20:41.970490 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.970460    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2c5c0e1-e434-493e-be41-f8e966a03648-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2c5c0e1-e434-493e-be41-f8e966a03648" (UID: "e2c5c0e1-e434-493e-be41-f8e966a03648"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:41.972112 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:41.972092    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2c5c0e1-e434-493e-be41-f8e966a03648-kube-api-access-2252k" (OuterVolumeSpecName: "kube-api-access-2252k") pod "e2c5c0e1-e434-493e-be41-f8e966a03648" (UID: "e2c5c0e1-e434-493e-be41-f8e966a03648"). InnerVolumeSpecName "kube-api-access-2252k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:42.071696 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.071619    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2252k\" (UniqueName: \"kubernetes.io/projected/e2c5c0e1-e434-493e-be41-f8e966a03648-kube-api-access-2252k\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:20:42.071696 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.071668    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2c5c0e1-e434-493e-be41-f8e966a03648-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:20:42.609993 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.609959    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"
Sep 14 08:20:42.609993 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.609968    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d" event={"ID":"1e91a6a2-9aaa-47b8-9960-b417ba260f7a","Type":"ContainerDied","Data":"05b7107dadf76a82acbea1bd01c97fb30f891065a0cdab990a3e07dae03733ec"}
Sep 14 08:20:42.610524 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.610050    2563 scope.go:117] "RemoveContainer" containerID="38676bf04b758469e04fb9a58ad4bc6ba9e81a4a8e7d13afd7130a96d74d5c92"
Sep 14 08:20:42.611457 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.611407    2563 generic.go:356] "Generic (PLEG): container finished" podID="e2c5c0e1-e434-493e-be41-f8e966a03648" containerID="99278a18a0bd895d9cea225f2f9353da8c09695be15d58373e7f5f485d7f30a8" exitCode=0
Sep 14 08:20:42.611527 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.611457    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"
Sep 14 08:20:42.611527 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.611493    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld" event={"ID":"e2c5c0e1-e434-493e-be41-f8e966a03648","Type":"ContainerDied","Data":"99278a18a0bd895d9cea225f2f9353da8c09695be15d58373e7f5f485d7f30a8"}
Sep 14 08:20:42.611635 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.611530    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld" event={"ID":"e2c5c0e1-e434-493e-be41-f8e966a03648","Type":"ContainerDied","Data":"01bc3a687f6ab505814fc5c6fe86941b09694b93e90a92df45ff76356d8e9f60"}
Sep 14 08:20:42.619511 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.619491    2563 scope.go:117] "RemoveContainer" containerID="99278a18a0bd895d9cea225f2f9353da8c09695be15d58373e7f5f485d7f30a8"
Sep 14 08:20:42.628379 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.628355    2563 scope.go:117] "RemoveContainer" containerID="99278a18a0bd895d9cea225f2f9353da8c09695be15d58373e7f5f485d7f30a8"
Sep 14 08:20:42.628663 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:20:42.628619    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99278a18a0bd895d9cea225f2f9353da8c09695be15d58373e7f5f485d7f30a8\": container with ID starting with 99278a18a0bd895d9cea225f2f9353da8c09695be15d58373e7f5f485d7f30a8 not found: ID does not exist" containerID="99278a18a0bd895d9cea225f2f9353da8c09695be15d58373e7f5f485d7f30a8"
Sep 14 08:20:42.628757 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.628675    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99278a18a0bd895d9cea225f2f9353da8c09695be15d58373e7f5f485d7f30a8"} err="failed to get container status \"99278a18a0bd895d9cea225f2f9353da8c09695be15d58373e7f5f485d7f30a8\": rpc error: code = NotFound desc = could not find container \"99278a18a0bd895d9cea225f2f9353da8c09695be15d58373e7f5f485d7f30a8\": container with ID starting with 99278a18a0bd895d9cea225f2f9353da8c09695be15d58373e7f5f485d7f30a8 not found: ID does not exist"
Sep 14 08:20:42.628944 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.628931    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"]
Sep 14 08:20:42.632504 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.632484    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7746e4-6cb44fcd5c-ssh4d"]
Sep 14 08:20:42.644009 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.643989    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"]
Sep 14 08:20:42.648698 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:42.648677    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7746e4-57f88dfb46-k4nld"]
Sep 14 08:20:43.885699 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:43.885663    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e91a6a2-9aaa-47b8-9960-b417ba260f7a" path="/var/lib/kubelet/pods/1e91a6a2-9aaa-47b8-9960-b417ba260f7a/volumes"
Sep 14 08:20:43.886080 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:43.886037    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2c5c0e1-e434-493e-be41-f8e966a03648" path="/var/lib/kubelet/pods/e2c5c0e1-e434-493e-be41-f8e966a03648/volumes"
Sep 14 08:20:48.802060 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.802023    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67"]
Sep 14 08:20:48.802454 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.802428    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e91a6a2-9aaa-47b8-9960-b417ba260f7a" containerName="payload-processing"
Sep 14 08:20:48.802454 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.802441    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e91a6a2-9aaa-47b8-9960-b417ba260f7a" containerName="payload-processing"
Sep 14 08:20:48.802454 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.802455    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8" containerName="revoke-keys"
Sep 14 08:20:48.802573 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.802462    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8" containerName="revoke-keys"
Sep 14 08:20:48.802573 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.802486    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2c5c0e1-e434-493e-be41-f8e966a03648" containerName="payload-pre-processing"
Sep 14 08:20:48.802573 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.802491    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2c5c0e1-e434-493e-be41-f8e966a03648" containerName="payload-pre-processing"
Sep 14 08:20:48.802573 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.802548    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d3f3528-8780-4ea3-9dd4-1cc72ea4e1f8" containerName="revoke-keys"
Sep 14 08:20:48.802573 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.802558    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e91a6a2-9aaa-47b8-9960-b417ba260f7a" containerName="payload-processing"
Sep 14 08:20:48.802573 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.802568    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2c5c0e1-e434-493e-be41-f8e966a03648" containerName="payload-pre-processing"
Sep 14 08:20:48.807032 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.807014    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67"
Sep 14 08:20:48.809338 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.809312    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:20:48.812544 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.812526    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67"]
Sep 14 08:20:48.930816 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:48.930780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1f28d0-054b-4178-90cd-f4864d49b449-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67\" (UID: \"7f1f28d0-054b-4178-90cd-f4864d49b449\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67"
Sep 14 08:20:49.032058 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:49.032020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1f28d0-054b-4178-90cd-f4864d49b449-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67\" (UID: \"7f1f28d0-054b-4178-90cd-f4864d49b449\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67"
Sep 14 08:20:49.032677 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:49.032632    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1f28d0-054b-4178-90cd-f4864d49b449-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67\" (UID: \"7f1f28d0-054b-4178-90cd-f4864d49b449\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67"
Sep 14 08:20:49.120033 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:49.120001    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67"
Sep 14 08:20:49.387890 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:49.387862    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67"]
Sep 14 08:20:49.388934 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:20:49.388909    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f1f28d0_054b_4178_90cd_f4864d49b449.slice/crio-d4d6a8b4efa6a8cc4e52ce39c6c657a700475e9dd722a28f1526efb18038f7e7 WatchSource:0}: Error finding container d4d6a8b4efa6a8cc4e52ce39c6c657a700475e9dd722a28f1526efb18038f7e7: Status 404 returned error can't find the container with id d4d6a8b4efa6a8cc4e52ce39c6c657a700475e9dd722a28f1526efb18038f7e7
Sep 14 08:20:49.654766 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:49.654680    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67" event={"ID":"7f1f28d0-054b-4178-90cd-f4864d49b449","Type":"ContainerStarted","Data":"1195d60da1378f8ca2d9d217c50a9a220a3515dd0451dd1ba422b3850eb48f1b"}
Sep 14 08:20:49.654766 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:49.654732    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67" event={"ID":"7f1f28d0-054b-4178-90cd-f4864d49b449","Type":"ContainerStarted","Data":"d4d6a8b4efa6a8cc4e52ce39c6c657a700475e9dd722a28f1526efb18038f7e7"}
Sep 14 08:20:49.669892 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:49.669843    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67" podStartSLOduration=1.6698269 podStartE2EDuration="1.6698269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:20:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:20:49.668423511 +0000 UTC m=+1970.297311138" watchObservedRunningTime="2026-09-14 08:20:49.6698269 +0000 UTC m=+1970.298714527"
Sep 14 08:20:50.662271 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:50.662236    2563 generic.go:356] "Generic (PLEG): container finished" podID="7f1f28d0-054b-4178-90cd-f4864d49b449" containerID="1195d60da1378f8ca2d9d217c50a9a220a3515dd0451dd1ba422b3850eb48f1b" exitCode=0
Sep 14 08:20:50.662681 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:50.662323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67" event={"ID":"7f1f28d0-054b-4178-90cd-f4864d49b449","Type":"ContainerDied","Data":"1195d60da1378f8ca2d9d217c50a9a220a3515dd0451dd1ba422b3850eb48f1b"}
Sep 14 08:20:51.797713 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:51.797686    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67"
Sep 14 08:20:51.855568 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:51.855539    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1f28d0-054b-4178-90cd-f4864d49b449-maas-api-service-ca\") pod \"7f1f28d0-054b-4178-90cd-f4864d49b449\" (UID: \"7f1f28d0-054b-4178-90cd-f4864d49b449\") "
Sep 14 08:20:51.855890 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:51.855869    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f1f28d0-054b-4178-90cd-f4864d49b449-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7f1f28d0-054b-4178-90cd-f4864d49b449" (UID: "7f1f28d0-054b-4178-90cd-f4864d49b449"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:51.956849 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:51.956764    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f1f28d0-054b-4178-90cd-f4864d49b449-maas-api-service-ca\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:20:52.675077 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:52.675043    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67"
Sep 14 08:20:52.675245 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:52.675075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67" event={"ID":"7f1f28d0-054b-4178-90cd-f4864d49b449","Type":"ContainerDied","Data":"d4d6a8b4efa6a8cc4e52ce39c6c657a700475e9dd722a28f1526efb18038f7e7"}
Sep 14 08:20:52.675245 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:52.675125    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4d6a8b4efa6a8cc4e52ce39c6c657a700475e9dd722a28f1526efb18038f7e7"
Sep 14 08:20:59.337693 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.337658    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ea9d35-29822895-8mx2s"]
Sep 14 08:20:59.340085 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.340051    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-ea9d35-29822895-8mx2s"]
Sep 14 08:20:59.361595 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.361560    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"]
Sep 14 08:20:59.369123 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.369086    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"]
Sep 14 08:20:59.369341 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.369287    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b" podUID="31cd80ad-a6a6-4cda-8368-4c06e8e25532" containerName="payload-pre-processing" containerID="cri-o://e221d2a87bd793999187c86841781277f1797c5c8702b84fcfc9c162e95ffa50" gracePeriod=30
Sep 14 08:20:59.530272 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.530221    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67"]
Sep 14 08:20:59.532656 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.532615    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-ea9d35-kgr67"]
Sep 14 08:20:59.632940 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.632917    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"
Sep 14 08:20:59.719708 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.719674    2563 generic.go:356] "Generic (PLEG): container finished" podID="31cd80ad-a6a6-4cda-8368-4c06e8e25532" containerID="e221d2a87bd793999187c86841781277f1797c5c8702b84fcfc9c162e95ffa50" exitCode=0
Sep 14 08:20:59.719860 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.719724    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"
Sep 14 08:20:59.719860 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.719761    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b" event={"ID":"31cd80ad-a6a6-4cda-8368-4c06e8e25532","Type":"ContainerDied","Data":"e221d2a87bd793999187c86841781277f1797c5c8702b84fcfc9c162e95ffa50"}
Sep 14 08:20:59.719860 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.719839    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b" event={"ID":"31cd80ad-a6a6-4cda-8368-4c06e8e25532","Type":"ContainerDied","Data":"ff20fb16b07b9928c55a7ff00665d93defc693500986a016334bbc4d97b0a100"}
Sep 14 08:20:59.719860 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.719861    2563 scope.go:117] "RemoveContainer" containerID="e221d2a87bd793999187c86841781277f1797c5c8702b84fcfc9c162e95ffa50"
Sep 14 08:20:59.720086 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.720014    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf" podUID="86ebb312-4a36-4534-b09c-73ae4391ca27" containerName="payload-processing" containerID="cri-o://64f75e4cfdd236f1f682a17ef4a11e25aa2e9d6e81eadbadbddf107da64cbfb6" gracePeriod=30
Sep 14 08:20:59.727193 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.727168    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8glj\" (UniqueName: \"kubernetes.io/projected/31cd80ad-a6a6-4cda-8368-4c06e8e25532-kube-api-access-x8glj\") pod \"31cd80ad-a6a6-4cda-8368-4c06e8e25532\" (UID: \"31cd80ad-a6a6-4cda-8368-4c06e8e25532\") "
Sep 14 08:20:59.727302 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.727230    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31cd80ad-a6a6-4cda-8368-4c06e8e25532-plugins-config-volume\") pod \"31cd80ad-a6a6-4cda-8368-4c06e8e25532\" (UID: \"31cd80ad-a6a6-4cda-8368-4c06e8e25532\") "
Sep 14 08:20:59.727818 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.727789    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31cd80ad-a6a6-4cda-8368-4c06e8e25532-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "31cd80ad-a6a6-4cda-8368-4c06e8e25532" (UID: "31cd80ad-a6a6-4cda-8368-4c06e8e25532"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:59.729468 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.729442    2563 scope.go:117] "RemoveContainer" containerID="e221d2a87bd793999187c86841781277f1797c5c8702b84fcfc9c162e95ffa50"
Sep 14 08:20:59.729574 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.729535    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31cd80ad-a6a6-4cda-8368-4c06e8e25532-kube-api-access-x8glj" (OuterVolumeSpecName: "kube-api-access-x8glj") pod "31cd80ad-a6a6-4cda-8368-4c06e8e25532" (UID: "31cd80ad-a6a6-4cda-8368-4c06e8e25532"). InnerVolumeSpecName "kube-api-access-x8glj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:59.729784 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:20:59.729762    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e221d2a87bd793999187c86841781277f1797c5c8702b84fcfc9c162e95ffa50\": container with ID starting with e221d2a87bd793999187c86841781277f1797c5c8702b84fcfc9c162e95ffa50 not found: ID does not exist" containerID="e221d2a87bd793999187c86841781277f1797c5c8702b84fcfc9c162e95ffa50"
Sep 14 08:20:59.729841 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.729792    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e221d2a87bd793999187c86841781277f1797c5c8702b84fcfc9c162e95ffa50"} err="failed to get container status \"e221d2a87bd793999187c86841781277f1797c5c8702b84fcfc9c162e95ffa50\": rpc error: code = NotFound desc = could not find container \"e221d2a87bd793999187c86841781277f1797c5c8702b84fcfc9c162e95ffa50\": container with ID starting with e221d2a87bd793999187c86841781277f1797c5c8702b84fcfc9c162e95ffa50 not found: ID does not exist"
Sep 14 08:20:59.828604 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.828572    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8glj\" (UniqueName: \"kubernetes.io/projected/31cd80ad-a6a6-4cda-8368-4c06e8e25532-kube-api-access-x8glj\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:20:59.828604 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.828598    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31cd80ad-a6a6-4cda-8368-4c06e8e25532-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:20:59.887534 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.887461    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f1f28d0-054b-4178-90cd-f4864d49b449" path="/var/lib/kubelet/pods/7f1f28d0-054b-4178-90cd-f4864d49b449/volumes"
Sep 14 08:20:59.887800 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.887786    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f475afb8-f976-439d-aa81-95563a8bfb7a" path="/var/lib/kubelet/pods/f475afb8-f976-439d-aa81-95563a8bfb7a/volumes"
Sep 14 08:20:59.985158 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:20:59.985128    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"
Sep 14 08:21:00.030851 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.030818    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcn4k\" (UniqueName: \"kubernetes.io/projected/86ebb312-4a36-4534-b09c-73ae4391ca27-kube-api-access-lcn4k\") pod \"86ebb312-4a36-4534-b09c-73ae4391ca27\" (UID: \"86ebb312-4a36-4534-b09c-73ae4391ca27\") "
Sep 14 08:21:00.031023 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.030861    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86ebb312-4a36-4534-b09c-73ae4391ca27-plugins-config-volume\") pod \"86ebb312-4a36-4534-b09c-73ae4391ca27\" (UID: \"86ebb312-4a36-4534-b09c-73ae4391ca27\") "
Sep 14 08:21:00.031282 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.031257    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86ebb312-4a36-4534-b09c-73ae4391ca27-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "86ebb312-4a36-4534-b09c-73ae4391ca27" (UID: "86ebb312-4a36-4534-b09c-73ae4391ca27"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:21:00.033284 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.033250    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86ebb312-4a36-4534-b09c-73ae4391ca27-kube-api-access-lcn4k" (OuterVolumeSpecName: "kube-api-access-lcn4k") pod "86ebb312-4a36-4534-b09c-73ae4391ca27" (UID: "86ebb312-4a36-4534-b09c-73ae4391ca27"). InnerVolumeSpecName "kube-api-access-lcn4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:21:00.035751 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.035725    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"]
Sep 14 08:21:00.039024 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.039002    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ea9d35-5fff6d67c4-tfg6b"]
Sep 14 08:21:00.132222 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.132193    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcn4k\" (UniqueName: \"kubernetes.io/projected/86ebb312-4a36-4534-b09c-73ae4391ca27-kube-api-access-lcn4k\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:21:00.132222 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.132218    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86ebb312-4a36-4534-b09c-73ae4391ca27-plugins-config-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:21:00.297687 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.297596    2563 scope.go:117] "RemoveContainer" containerID="64f75e4cfdd236f1f682a17ef4a11e25aa2e9d6e81eadbadbddf107da64cbfb6"
Sep 14 08:21:00.725682 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.725636    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf" event={"ID":"86ebb312-4a36-4534-b09c-73ae4391ca27","Type":"ContainerDied","Data":"64f75e4cfdd236f1f682a17ef4a11e25aa2e9d6e81eadbadbddf107da64cbfb6"}
Sep 14 08:21:00.726075 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.725705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf" event={"ID":"86ebb312-4a36-4534-b09c-73ae4391ca27","Type":"ContainerDied","Data":"62e322d883bf098a6209a8be0d05aa86567e8750d377e7f60f69d2fb3222b7aa"}
Sep 14 08:21:00.726075 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.725708    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"
Sep 14 08:21:00.757352 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.757315    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"]
Sep 14 08:21:00.761190 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:00.761164    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-ea9d35-7cc7b7f559-gzsbf"]
Sep 14 08:21:01.888279 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:01.888246    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31cd80ad-a6a6-4cda-8368-4c06e8e25532" path="/var/lib/kubelet/pods/31cd80ad-a6a6-4cda-8368-4c06e8e25532/volumes"
Sep 14 08:21:01.888658 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:01.888606    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86ebb312-4a36-4534-b09c-73ae4391ca27" path="/var/lib/kubelet/pods/86ebb312-4a36-4534-b09c-73ae4391ca27/volumes"
Sep 14 08:21:08.752527 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:08.752487    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6945fd44b-qgvbb"]
Sep 14 08:21:08.753035 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:08.752741    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6945fd44b-qgvbb" podUID="c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be" containerName="manager" containerID="cri-o://62c567877446d4bc7a6dcb73d21241bf918ec2a68b732c42f042fd83e07ae5de" gracePeriod=30
Sep 14 08:21:08.989589 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:08.989565    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 08:21:09.112429 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.112405    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-webhook-cert\") pod \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\" (UID: \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\") "
Sep 14 08:21:09.112783 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.112484    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-metrics-certs\") pod \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\" (UID: \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\") "
Sep 14 08:21:09.112783 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.112552    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ct7d9\" (UniqueName: \"kubernetes.io/projected/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-kube-api-access-ct7d9\") pod \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\" (UID: \"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be\") "
Sep 14 08:21:09.114612 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.114539    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be" (UID: "c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:21:09.114612 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.114560    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-kube-api-access-ct7d9" (OuterVolumeSpecName: "kube-api-access-ct7d9") pod "c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be" (UID: "c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be"). InnerVolumeSpecName "kube-api-access-ct7d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:21:09.114820 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.114732    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be" (UID: "c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:21:09.213994 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.213968    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-metrics-certs\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:21:09.213994 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.213990    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ct7d9\" (UniqueName: \"kubernetes.io/projected/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-kube-api-access-ct7d9\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:21:09.214130 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.213999    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be-webhook-cert\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:21:09.778099 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.778060    2563 generic.go:356] "Generic (PLEG): container finished" podID="c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be" containerID="62c567877446d4bc7a6dcb73d21241bf918ec2a68b732c42f042fd83e07ae5de" exitCode=0
Sep 14 08:21:09.778099 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.778100    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6945fd44b-qgvbb"
Sep 14 08:21:09.778621 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.778118    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6945fd44b-qgvbb" event={"ID":"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be","Type":"ContainerDied","Data":"62c567877446d4bc7a6dcb73d21241bf918ec2a68b732c42f042fd83e07ae5de"}
Sep 14 08:21:09.778621 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.778168    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6945fd44b-qgvbb" event={"ID":"c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be","Type":"ContainerDied","Data":"7c97295e7cdcbe099160f552775fbe7aa3539e42d9531b1b88288768c51b6553"}
Sep 14 08:21:09.778621 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.778202    2563 scope.go:117] "RemoveContainer" containerID="62c567877446d4bc7a6dcb73d21241bf918ec2a68b732c42f042fd83e07ae5de"
Sep 14 08:21:09.788601 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.788582    2563 scope.go:117] "RemoveContainer" containerID="62c567877446d4bc7a6dcb73d21241bf918ec2a68b732c42f042fd83e07ae5de"
Sep 14 08:21:09.788888 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:21:09.788867    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62c567877446d4bc7a6dcb73d21241bf918ec2a68b732c42f042fd83e07ae5de\": container with ID starting with 62c567877446d4bc7a6dcb73d21241bf918ec2a68b732c42f042fd83e07ae5de not found: ID does not exist" containerID="62c567877446d4bc7a6dcb73d21241bf918ec2a68b732c42f042fd83e07ae5de"
Sep 14 08:21:09.788984 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.788898    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62c567877446d4bc7a6dcb73d21241bf918ec2a68b732c42f042fd83e07ae5de"} err="failed to get container status \"62c567877446d4bc7a6dcb73d21241bf918ec2a68b732c42f042fd83e07ae5de\": rpc error: code = NotFound desc = could not find container \"62c567877446d4bc7a6dcb73d21241bf918ec2a68b732c42f042fd83e07ae5de\": container with ID starting with 62c567877446d4bc7a6dcb73d21241bf918ec2a68b732c42f042fd83e07ae5de not found: ID does not exist"
Sep 14 08:21:09.798688 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.798664    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6945fd44b-qgvbb"]
Sep 14 08:21:09.809609 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.804543    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6945fd44b-qgvbb"]
Sep 14 08:21:09.885271 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:21:09.885232    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be" path="/var/lib/kubelet/pods/c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be/volumes"
Sep 14 08:22:00.332021 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:00.331990    2563 scope.go:117] "RemoveContainer" containerID="85b3e5a1266cf2ba5fef4ae594cc4fedf8afe2581cdfea76d63c1c9550550d2f"
Sep 14 08:22:10.598803 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.598773    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6945fd44b-dhx7t"]
Sep 14 08:22:10.599349 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.599150    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31cd80ad-a6a6-4cda-8368-4c06e8e25532" containerName="payload-pre-processing"
Sep 14 08:22:10.599349 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.599164    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="31cd80ad-a6a6-4cda-8368-4c06e8e25532" containerName="payload-pre-processing"
Sep 14 08:22:10.599349 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.599175    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86ebb312-4a36-4534-b09c-73ae4391ca27" containerName="payload-processing"
Sep 14 08:22:10.599349 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.599183    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="86ebb312-4a36-4534-b09c-73ae4391ca27" containerName="payload-processing"
Sep 14 08:22:10.599349 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.599204    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be" containerName="manager"
Sep 14 08:22:10.599349 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.599210    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be" containerName="manager"
Sep 14 08:22:10.599349 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.599219    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f1f28d0-054b-4178-90cd-f4864d49b449" containerName="revoke-keys"
Sep 14 08:22:10.599349 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.599224    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f1f28d0-054b-4178-90cd-f4864d49b449" containerName="revoke-keys"
Sep 14 08:22:10.599349 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.599296    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f1f28d0-054b-4178-90cd-f4864d49b449" containerName="revoke-keys"
Sep 14 08:22:10.599349 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.599310    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="86ebb312-4a36-4534-b09c-73ae4391ca27" containerName="payload-processing"
Sep 14 08:22:10.599349 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.599319    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6f1dab9-82b8-49aa-8ea8-1f9c4abb88be" containerName="manager"
Sep 14 08:22:10.599349 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.599327    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="31cd80ad-a6a6-4cda-8368-4c06e8e25532" containerName="payload-pre-processing"
Sep 14 08:22:10.602335 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.602319    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:10.605158 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.605134    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 08:22:10.605338 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.605172    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5j4xp\""
Sep 14 08:22:10.605338 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.605175    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 08:22:10.605338 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.605174    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 08:22:10.613562 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.613542    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6945fd44b-dhx7t"]
Sep 14 08:22:10.732942 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.732916    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-webhook-cert\") pod \"maas-controller-6945fd44b-dhx7t\" (UID: \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\") " pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:10.733087 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.732955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj997\" (UniqueName: \"kubernetes.io/projected/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-kube-api-access-sj997\") pod \"maas-controller-6945fd44b-dhx7t\" (UID: \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\") " pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:10.733087 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.732976    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-metrics-certs\") pod \"maas-controller-6945fd44b-dhx7t\" (UID: \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\") " pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:10.833718 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.833677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-webhook-cert\") pod \"maas-controller-6945fd44b-dhx7t\" (UID: \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\") " pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:10.833718 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.833729    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj997\" (UniqueName: \"kubernetes.io/projected/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-kube-api-access-sj997\") pod \"maas-controller-6945fd44b-dhx7t\" (UID: \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\") " pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:10.833889 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.833749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-metrics-certs\") pod \"maas-controller-6945fd44b-dhx7t\" (UID: \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\") " pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:10.836117 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.836091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-metrics-certs\") pod \"maas-controller-6945fd44b-dhx7t\" (UID: \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\") " pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:10.836253 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.836231    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-webhook-cert\") pod \"maas-controller-6945fd44b-dhx7t\" (UID: \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\") " pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:10.841457 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.841435    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj997\" (UniqueName: \"kubernetes.io/projected/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-kube-api-access-sj997\") pod \"maas-controller-6945fd44b-dhx7t\" (UID: \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\") " pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:10.913796 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:10.913728    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:11.183301 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:11.183275    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6945fd44b-dhx7t"]
Sep 14 08:22:11.185795 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:22:11.185766    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac66976f_13e4_40fe_a7f7_1e1be9d6486a.slice/crio-360f853ecbeadd18af883bcd8af40ee76e0bf55a92c74efdf3cacd5d8fc65bef WatchSource:0}: Error finding container 360f853ecbeadd18af883bcd8af40ee76e0bf55a92c74efdf3cacd5d8fc65bef: Status 404 returned error can't find the container with id 360f853ecbeadd18af883bcd8af40ee76e0bf55a92c74efdf3cacd5d8fc65bef
Sep 14 08:22:11.187315 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:11.187291    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:22:12.131318 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:12.131279    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6945fd44b-dhx7t" event={"ID":"ac66976f-13e4-40fe-a7f7-1e1be9d6486a","Type":"ContainerStarted","Data":"511ebe1f245d99c58acfaf6a4557f7e32910f9c8e6294e65754c898b32df15c5"}
Sep 14 08:22:12.131716 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:12.131336    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6945fd44b-dhx7t" event={"ID":"ac66976f-13e4-40fe-a7f7-1e1be9d6486a","Type":"ContainerStarted","Data":"360f853ecbeadd18af883bcd8af40ee76e0bf55a92c74efdf3cacd5d8fc65bef"}
Sep 14 08:22:12.131716 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:12.131468    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:12.150634 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:12.150585    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6945fd44b-dhx7t" podStartSLOduration=1.518837818 podStartE2EDuration="2.150570892s" totalImagesPullingTime="631.733074ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:22:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:22:12.14829869 +0000 UTC m=+2052.777186342" watchObservedRunningTime="2026-09-14 08:22:12.150570892 +0000 UTC m=+2052.779458548"
Sep 14 08:22:23.139509 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:23.139433    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:32.479877 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:32.479828    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6945fd44b-dhx7t"]
Sep 14 08:22:32.480534 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:32.480037    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6945fd44b-dhx7t" podUID="ac66976f-13e4-40fe-a7f7-1e1be9d6486a" containerName="manager" containerID="cri-o://511ebe1f245d99c58acfaf6a4557f7e32910f9c8e6294e65754c898b32df15c5" gracePeriod=30
Sep 14 08:22:32.729037 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:32.729009    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:32.825968 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:32.825876    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-webhook-cert\") pod \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\" (UID: \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\") "
Sep 14 08:22:32.825968 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:32.825965    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sj997\" (UniqueName: \"kubernetes.io/projected/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-kube-api-access-sj997\") pod \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\" (UID: \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\") "
Sep 14 08:22:32.826188 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:32.825998    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-metrics-certs\") pod \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\" (UID: \"ac66976f-13e4-40fe-a7f7-1e1be9d6486a\") "
Sep 14 08:22:32.827923 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:32.827884    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ac66976f-13e4-40fe-a7f7-1e1be9d6486a" (UID: "ac66976f-13e4-40fe-a7f7-1e1be9d6486a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:22:32.828070 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:32.828042    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-kube-api-access-sj997" (OuterVolumeSpecName: "kube-api-access-sj997") pod "ac66976f-13e4-40fe-a7f7-1e1be9d6486a" (UID: "ac66976f-13e4-40fe-a7f7-1e1be9d6486a"). InnerVolumeSpecName "kube-api-access-sj997". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:22:32.828263 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:32.828242    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ac66976f-13e4-40fe-a7f7-1e1be9d6486a" (UID: "ac66976f-13e4-40fe-a7f7-1e1be9d6486a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:22:32.927363 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:32.927329    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj997\" (UniqueName: \"kubernetes.io/projected/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-kube-api-access-sj997\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:22:32.927363 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:32.927357    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-metrics-certs\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:22:32.927363 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:32.927367    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac66976f-13e4-40fe-a7f7-1e1be9d6486a-webhook-cert\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:22:33.252649 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:33.252615    2563 generic.go:356] "Generic (PLEG): container finished" podID="ac66976f-13e4-40fe-a7f7-1e1be9d6486a" containerID="511ebe1f245d99c58acfaf6a4557f7e32910f9c8e6294e65754c898b32df15c5" exitCode=0
Sep 14 08:22:33.252859 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:33.252674    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6945fd44b-dhx7t"
Sep 14 08:22:33.252859 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:33.252709    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6945fd44b-dhx7t" event={"ID":"ac66976f-13e4-40fe-a7f7-1e1be9d6486a","Type":"ContainerDied","Data":"511ebe1f245d99c58acfaf6a4557f7e32910f9c8e6294e65754c898b32df15c5"}
Sep 14 08:22:33.252859 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:33.252762    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6945fd44b-dhx7t" event={"ID":"ac66976f-13e4-40fe-a7f7-1e1be9d6486a","Type":"ContainerDied","Data":"360f853ecbeadd18af883bcd8af40ee76e0bf55a92c74efdf3cacd5d8fc65bef"}
Sep 14 08:22:33.252859 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:33.252786    2563 scope.go:117] "RemoveContainer" containerID="511ebe1f245d99c58acfaf6a4557f7e32910f9c8e6294e65754c898b32df15c5"
Sep 14 08:22:33.263837 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:33.263220    2563 scope.go:117] "RemoveContainer" containerID="511ebe1f245d99c58acfaf6a4557f7e32910f9c8e6294e65754c898b32df15c5"
Sep 14 08:22:33.265134 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:22:33.265111    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"511ebe1f245d99c58acfaf6a4557f7e32910f9c8e6294e65754c898b32df15c5\": container with ID starting with 511ebe1f245d99c58acfaf6a4557f7e32910f9c8e6294e65754c898b32df15c5 not found: ID does not exist" containerID="511ebe1f245d99c58acfaf6a4557f7e32910f9c8e6294e65754c898b32df15c5"
Sep 14 08:22:33.265253 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:33.265139    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511ebe1f245d99c58acfaf6a4557f7e32910f9c8e6294e65754c898b32df15c5"} err="failed to get container status \"511ebe1f245d99c58acfaf6a4557f7e32910f9c8e6294e65754c898b32df15c5\": rpc error: code = NotFound desc = could not find container \"511ebe1f245d99c58acfaf6a4557f7e32910f9c8e6294e65754c898b32df15c5\": container with ID starting with 511ebe1f245d99c58acfaf6a4557f7e32910f9c8e6294e65754c898b32df15c5 not found: ID does not exist"
Sep 14 08:22:33.275805 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:33.275783    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6945fd44b-dhx7t"]
Sep 14 08:22:33.281624 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:33.281597    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6945fd44b-dhx7t"]
Sep 14 08:22:33.885728 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:33.885694    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac66976f-13e4-40fe-a7f7-1e1be9d6486a" path="/var/lib/kubelet/pods/ac66976f-13e4-40fe-a7f7-1e1be9d6486a/volumes"
Sep 14 08:22:34.058898 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.058864    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6945fd44b-hndfp"]
Sep 14 08:22:34.059304 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.059258    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac66976f-13e4-40fe-a7f7-1e1be9d6486a" containerName="manager"
Sep 14 08:22:34.059304 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.059296    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac66976f-13e4-40fe-a7f7-1e1be9d6486a" containerName="manager"
Sep 14 08:22:34.059519 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.059414    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac66976f-13e4-40fe-a7f7-1e1be9d6486a" containerName="manager"
Sep 14 08:22:34.062405 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.062387    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6945fd44b-hndfp"
Sep 14 08:22:34.065163 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.065143    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 08:22:34.065283 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.065161    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 08:22:34.065353 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.065211    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5j4xp\""
Sep 14 08:22:34.065419 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.065222    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 08:22:34.071469 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.071446    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6945fd44b-hndfp"]
Sep 14 08:22:34.239056 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.238967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/babff1f3-9f42-40f7-8112-3c9be4f3e13b-webhook-cert\") pod \"maas-controller-6945fd44b-hndfp\" (UID: \"babff1f3-9f42-40f7-8112-3c9be4f3e13b\") " pod="opendatahub/maas-controller-6945fd44b-hndfp"
Sep 14 08:22:34.239056 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.239009    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/babff1f3-9f42-40f7-8112-3c9be4f3e13b-metrics-certs\") pod \"maas-controller-6945fd44b-hndfp\" (UID: \"babff1f3-9f42-40f7-8112-3c9be4f3e13b\") " pod="opendatahub/maas-controller-6945fd44b-hndfp"
Sep 14 08:22:34.239263 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.239057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmmd8\" (UniqueName: \"kubernetes.io/projected/babff1f3-9f42-40f7-8112-3c9be4f3e13b-kube-api-access-jmmd8\") pod \"maas-controller-6945fd44b-hndfp\" (UID: \"babff1f3-9f42-40f7-8112-3c9be4f3e13b\") " pod="opendatahub/maas-controller-6945fd44b-hndfp"
Sep 14 08:22:34.340326 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.340292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/babff1f3-9f42-40f7-8112-3c9be4f3e13b-webhook-cert\") pod \"maas-controller-6945fd44b-hndfp\" (UID: \"babff1f3-9f42-40f7-8112-3c9be4f3e13b\") " pod="opendatahub/maas-controller-6945fd44b-hndfp"
Sep 14 08:22:34.340528 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.340343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/babff1f3-9f42-40f7-8112-3c9be4f3e13b-metrics-certs\") pod \"maas-controller-6945fd44b-hndfp\" (UID: \"babff1f3-9f42-40f7-8112-3c9be4f3e13b\") " pod="opendatahub/maas-controller-6945fd44b-hndfp"
Sep 14 08:22:34.340528 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.340395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmmd8\" (UniqueName: \"kubernetes.io/projected/babff1f3-9f42-40f7-8112-3c9be4f3e13b-kube-api-access-jmmd8\") pod \"maas-controller-6945fd44b-hndfp\" (UID: \"babff1f3-9f42-40f7-8112-3c9be4f3e13b\") " pod="opendatahub/maas-controller-6945fd44b-hndfp"
Sep 14 08:22:34.342961 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.342936    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/babff1f3-9f42-40f7-8112-3c9be4f3e13b-webhook-cert\") pod \"maas-controller-6945fd44b-hndfp\" (UID: \"babff1f3-9f42-40f7-8112-3c9be4f3e13b\") " pod="opendatahub/maas-controller-6945fd44b-hndfp"
Sep 14 08:22:34.343072 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.343049    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/babff1f3-9f42-40f7-8112-3c9be4f3e13b-metrics-certs\") pod \"maas-controller-6945fd44b-hndfp\" (UID: \"babff1f3-9f42-40f7-8112-3c9be4f3e13b\") " pod="opendatahub/maas-controller-6945fd44b-hndfp"
Sep 14 08:22:34.348977 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.348951    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmmd8\" (UniqueName: \"kubernetes.io/projected/babff1f3-9f42-40f7-8112-3c9be4f3e13b-kube-api-access-jmmd8\") pod \"maas-controller-6945fd44b-hndfp\" (UID: \"babff1f3-9f42-40f7-8112-3c9be4f3e13b\") " pod="opendatahub/maas-controller-6945fd44b-hndfp"
Sep 14 08:22:34.373422 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.373402    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6945fd44b-hndfp"
Sep 14 08:22:34.646816 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:34.646789    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6945fd44b-hndfp"]
Sep 14 08:22:34.648150 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:22:34.648114    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbabff1f3_9f42_40f7_8112_3c9be4f3e13b.slice/crio-96519bf0e1aaf473e3a48cdeb53d5f97cac5971cebf1a9602c2f37fafd6cc244 WatchSource:0}: Error finding container 96519bf0e1aaf473e3a48cdeb53d5f97cac5971cebf1a9602c2f37fafd6cc244: Status 404 returned error can't find the container with id 96519bf0e1aaf473e3a48cdeb53d5f97cac5971cebf1a9602c2f37fafd6cc244
Sep 14 08:22:35.267096 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:35.267059    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6945fd44b-hndfp" event={"ID":"babff1f3-9f42-40f7-8112-3c9be4f3e13b","Type":"ContainerStarted","Data":"b744366bef9c1ee6b913c89f325c1b79524b9db00123511b3c74d6c2a5780749"}
Sep 14 08:22:35.267439 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:35.267113    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6945fd44b-hndfp" event={"ID":"babff1f3-9f42-40f7-8112-3c9be4f3e13b","Type":"ContainerStarted","Data":"96519bf0e1aaf473e3a48cdeb53d5f97cac5971cebf1a9602c2f37fafd6cc244"}
Sep 14 08:22:35.267439 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:35.267248    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6945fd44b-hndfp"
Sep 14 08:22:35.284452 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:35.284399    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6945fd44b-hndfp" podStartSLOduration=0.762079199 podStartE2EDuration="1.28437959s" totalImagesPullingTime="522.300391ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:22:35.283495137 +0000 UTC m=+2075.912382797" watchObservedRunningTime="2026-09-14 08:22:35.28437959 +0000 UTC m=+2075.913267218"
Sep 14 08:22:46.276809 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:22:46.276779    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6945fd44b-hndfp"
Sep 14 08:23:00.015113 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:00.015086    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 08:23:00.020853 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:00.020826    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 08:23:50.225607 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.225570    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"]
Sep 14 08:23:50.226125 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.225787    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml" podUID="247a0b3c-d216-4ff2-9560-8dd722dc6412" containerName="manager" containerID="cri-o://41543f37d1b1a8ac29bbfa7b790ec77e71f17551e0e56220680c820e86813415" gracePeriod=10
Sep 14 08:23:50.472603 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.472580    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"
Sep 14 08:23:50.504972 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.504874    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkhds\" (UniqueName: \"kubernetes.io/projected/247a0b3c-d216-4ff2-9560-8dd722dc6412-kube-api-access-mkhds\") pod \"247a0b3c-d216-4ff2-9560-8dd722dc6412\" (UID: \"247a0b3c-d216-4ff2-9560-8dd722dc6412\") "
Sep 14 08:23:50.504972 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.504923    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/247a0b3c-d216-4ff2-9560-8dd722dc6412-extensions-socket-volume\") pod \"247a0b3c-d216-4ff2-9560-8dd722dc6412\" (UID: \"247a0b3c-d216-4ff2-9560-8dd722dc6412\") "
Sep 14 08:23:50.505562 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.505534    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/247a0b3c-d216-4ff2-9560-8dd722dc6412-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "247a0b3c-d216-4ff2-9560-8dd722dc6412" (UID: "247a0b3c-d216-4ff2-9560-8dd722dc6412"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:50.508053 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.508024    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/247a0b3c-d216-4ff2-9560-8dd722dc6412-kube-api-access-mkhds" (OuterVolumeSpecName: "kube-api-access-mkhds") pod "247a0b3c-d216-4ff2-9560-8dd722dc6412" (UID: "247a0b3c-d216-4ff2-9560-8dd722dc6412"). InnerVolumeSpecName "kube-api-access-mkhds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:50.606241 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.606210    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkhds\" (UniqueName: \"kubernetes.io/projected/247a0b3c-d216-4ff2-9560-8dd722dc6412-kube-api-access-mkhds\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:23:50.606241 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.606236    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/247a0b3c-d216-4ff2-9560-8dd722dc6412-extensions-socket-volume\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:23:50.687483 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.687454    2563 generic.go:356] "Generic (PLEG): container finished" podID="247a0b3c-d216-4ff2-9560-8dd722dc6412" containerID="41543f37d1b1a8ac29bbfa7b790ec77e71f17551e0e56220680c820e86813415" exitCode=0
Sep 14 08:23:50.687617 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.687496    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"
Sep 14 08:23:50.687617 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.687539    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml" event={"ID":"247a0b3c-d216-4ff2-9560-8dd722dc6412","Type":"ContainerDied","Data":"41543f37d1b1a8ac29bbfa7b790ec77e71f17551e0e56220680c820e86813415"}
Sep 14 08:23:50.687617 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.687589    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml" event={"ID":"247a0b3c-d216-4ff2-9560-8dd722dc6412","Type":"ContainerDied","Data":"a8156a871019a21301f3f37142a0a3c750388ee543b80009889d09362e4d3fc2"}
Sep 14 08:23:50.687617 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.687612    2563 scope.go:117] "RemoveContainer" containerID="41543f37d1b1a8ac29bbfa7b790ec77e71f17551e0e56220680c820e86813415"
Sep 14 08:23:50.697070 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.697044    2563 scope.go:117] "RemoveContainer" containerID="41543f37d1b1a8ac29bbfa7b790ec77e71f17551e0e56220680c820e86813415"
Sep 14 08:23:50.697334 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:23:50.697317    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41543f37d1b1a8ac29bbfa7b790ec77e71f17551e0e56220680c820e86813415\": container with ID starting with 41543f37d1b1a8ac29bbfa7b790ec77e71f17551e0e56220680c820e86813415 not found: ID does not exist" containerID="41543f37d1b1a8ac29bbfa7b790ec77e71f17551e0e56220680c820e86813415"
Sep 14 08:23:50.697393 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.697343    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41543f37d1b1a8ac29bbfa7b790ec77e71f17551e0e56220680c820e86813415"} err="failed to get container status \"41543f37d1b1a8ac29bbfa7b790ec77e71f17551e0e56220680c820e86813415\": rpc error: code = NotFound desc = could not find container \"41543f37d1b1a8ac29bbfa7b790ec77e71f17551e0e56220680c820e86813415\": container with ID starting with 41543f37d1b1a8ac29bbfa7b790ec77e71f17551e0e56220680c820e86813415 not found: ID does not exist"
Sep 14 08:23:50.711150 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.711124    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"]
Sep 14 08:23:50.715491 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:50.715471    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-f65ml"]
Sep 14 08:23:51.886096 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:23:51.886063    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="247a0b3c-d216-4ff2-9560-8dd722dc6412" path="/var/lib/kubelet/pods/247a0b3c-d216-4ff2-9560-8dd722dc6412/volumes"
Sep 14 08:24:50.914157 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:50.914117    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6"]
Sep 14 08:24:50.914592 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:50.914468    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="247a0b3c-d216-4ff2-9560-8dd722dc6412" containerName="manager"
Sep 14 08:24:50.914592 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:50.914479    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="247a0b3c-d216-4ff2-9560-8dd722dc6412" containerName="manager"
Sep 14 08:24:50.914592 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:50.914543    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="247a0b3c-d216-4ff2-9560-8dd722dc6412" containerName="manager"
Sep 14 08:24:50.918435 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:50.918420    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6"
Sep 14 08:24:50.920841 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:50.920820    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xpfct\""
Sep 14 08:24:50.929458 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:50.929436    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6"]
Sep 14 08:24:51.021970 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:51.021942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5h9h\" (UniqueName: \"kubernetes.io/projected/1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9-kube-api-access-b5h9h\") pod \"kuadrant-operator-controller-manager-5769f99d9f-25nq6\" (UID: \"1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6"
Sep 14 08:24:51.022132 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:51.021998    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-25nq6\" (UID: \"1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6"
Sep 14 08:24:51.122553 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:51.122525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-25nq6\" (UID: \"1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6"
Sep 14 08:24:51.122714 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:51.122585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5h9h\" (UniqueName: \"kubernetes.io/projected/1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9-kube-api-access-b5h9h\") pod \"kuadrant-operator-controller-manager-5769f99d9f-25nq6\" (UID: \"1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6"
Sep 14 08:24:51.122918 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:51.122897    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-25nq6\" (UID: \"1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6"
Sep 14 08:24:51.137121 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:51.137091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5h9h\" (UniqueName: \"kubernetes.io/projected/1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9-kube-api-access-b5h9h\") pod \"kuadrant-operator-controller-manager-5769f99d9f-25nq6\" (UID: \"1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6"
Sep 14 08:24:51.229245 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:51.229187    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6"
Sep 14 08:24:51.503833 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:51.503806    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6"]
Sep 14 08:24:51.504533 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:24:51.504509    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fdf9b58_e57a_4bc0_b1c0_51a5239c73e9.slice/crio-9a13aa89302acb41c9a5f2f6d0c5c1f08e97698b3523bf3e1445e1da9b22f5d5 WatchSource:0}: Error finding container 9a13aa89302acb41c9a5f2f6d0c5c1f08e97698b3523bf3e1445e1da9b22f5d5: Status 404 returned error can't find the container with id 9a13aa89302acb41c9a5f2f6d0c5c1f08e97698b3523bf3e1445e1da9b22f5d5
Sep 14 08:24:52.016702 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:52.016663    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6" event={"ID":"1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9","Type":"ContainerStarted","Data":"4739dffbbf8060804ea20e19012918dddd80e4717c9e3a1461d4aaa687e14d4e"}
Sep 14 08:24:52.017147 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:52.016719    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6"
Sep 14 08:24:52.017147 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:52.016739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6" event={"ID":"1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9","Type":"ContainerStarted","Data":"9a13aa89302acb41c9a5f2f6d0c5c1f08e97698b3523bf3e1445e1da9b22f5d5"}
Sep 14 08:24:52.039171 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:24:52.039074    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6" podStartSLOduration=2.03905586 podStartE2EDuration="2.03905586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:24:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:24:52.036974225 +0000 UTC m=+2212.665861853" watchObservedRunningTime="2026-09-14 08:24:52.03905586 +0000 UTC m=+2212.667943488"
Sep 14 08:25:00.382780 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:00.382751    2563 scope.go:117] "RemoveContainer" containerID="77774c99959cd1d15b69127a190ee7f493cf63063359ca6d04f2a23c4a61cac3"
Sep 14 08:25:03.024354 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:03.024323    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-25nq6"
Sep 14 08:25:44.401241 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.401207    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"]
Sep 14 08:25:44.404486 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.404468    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.407066 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.407046    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-96ce3f1c-kserve-self-signed-certs\""
Sep 14 08:25:44.414473 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.414451    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"]
Sep 14 08:25:44.503231 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.503043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-tmp-dir\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.503231 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.503132    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt69w\" (UniqueName: \"kubernetes.io/projected/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-kube-api-access-mt69w\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.503231 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.503157    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-model-cache\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.503231 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.503193    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-home\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.503588 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.503259    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-dshm\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.503588 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.503305    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-tls-certs\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.604102 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.604068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-dshm\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.604285 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.604132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-tls-certs\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.604285 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.604219    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-tmp-dir\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.604413 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.604283    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt69w\" (UniqueName: \"kubernetes.io/projected/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-kube-api-access-mt69w\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.604413 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.604314    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-model-cache\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.604413 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.604352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-home\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.604679 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.604655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-tmp-dir\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.604787 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.604720    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-home\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.604849 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.604821    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-model-cache\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.606910 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.606893    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-dshm\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.607293 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.607270    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-tls-certs\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.620246 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.620226    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt69w\" (UniqueName: \"kubernetes.io/projected/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-kube-api-access-mt69w\") pod \"e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") " pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.715279 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.715211    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:44.987834 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:44.987803    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"]
Sep 14 08:25:44.989087 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:25:44.989062    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded4bf1dd_8dc5_4d98_9596_57abd80b707f.slice/crio-57a859bb2d22cffe00e39bb01a20c6bd41647d1e53975f0562f2ac592d892d97 WatchSource:0}: Error finding container 57a859bb2d22cffe00e39bb01a20c6bd41647d1e53975f0562f2ac592d892d97: Status 404 returned error can't find the container with id 57a859bb2d22cffe00e39bb01a20c6bd41647d1e53975f0562f2ac592d892d97
Sep 14 08:25:45.317704 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:45.317617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw" event={"ID":"ed4bf1dd-8dc5-4d98-9596-57abd80b707f","Type":"ContainerStarted","Data":"c8915a2528849cfb5a6d398f62b7474877bd81a4500a25ba8f82687c526cb4d0"}
Sep 14 08:25:45.317704 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:45.317684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw" event={"ID":"ed4bf1dd-8dc5-4d98-9596-57abd80b707f","Type":"ContainerStarted","Data":"57a859bb2d22cffe00e39bb01a20c6bd41647d1e53975f0562f2ac592d892d97"}
Sep 14 08:25:45.334587 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:45.334526    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw" podStartSLOduration=1.3345127940000001 podStartE2EDuration="1.334512794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:25:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:25:45.333551486 +0000 UTC m=+2265.962439135" watchObservedRunningTime="2026-09-14 08:25:45.334512794 +0000 UTC m=+2265.963400419"
Sep 14 08:25:54.716008 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:54.715962    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:54.716468 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:54.716181    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:54.728745 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:54.728724    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:25:55.387340 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:25:55.387302    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:26:00.402453 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:00.402415    2563 scope.go:117] "RemoveContainer" containerID="e4853d69e7c802c3ff5c00f40817e924605b931b95adebd0f988181951d91aca"
Sep 14 08:26:14.582936 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:14.582899    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"]
Sep 14 08:26:29.602884 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:29.602832    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw" podUID="ed4bf1dd-8dc5-4d98-9596-57abd80b707f" containerName="main" containerID="cri-o://c8915a2528849cfb5a6d398f62b7474877bd81a4500a25ba8f82687c526cb4d0" gracePeriod=45
Sep 14 08:26:29.864360 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:29.864291    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:26:30.018894 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.018863    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-dshm\") pod \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") "
Sep 14 08:26:30.018894 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.018900    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-model-cache\") pod \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") "
Sep 14 08:26:30.019132 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.018947    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-tmp-dir\") pod \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") "
Sep 14 08:26:30.019132 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.018998    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-tls-certs\") pod \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") "
Sep 14 08:26:30.019132 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.019031    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-home\") pod \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") "
Sep 14 08:26:30.019132 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.019086    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt69w\" (UniqueName: \"kubernetes.io/projected/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-kube-api-access-mt69w\") pod \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\" (UID: \"ed4bf1dd-8dc5-4d98-9596-57abd80b707f\") "
Sep 14 08:26:30.019348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.019231    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed4bf1dd-8dc5-4d98-9596-57abd80b707f" (UID: "ed4bf1dd-8dc5-4d98-9596-57abd80b707f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:26:30.019348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.019250    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed4bf1dd-8dc5-4d98-9596-57abd80b707f" (UID: "ed4bf1dd-8dc5-4d98-9596-57abd80b707f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:26:30.019348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.019317    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-home" (OuterVolumeSpecName: "home") pod "ed4bf1dd-8dc5-4d98-9596-57abd80b707f" (UID: "ed4bf1dd-8dc5-4d98-9596-57abd80b707f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:26:30.019500 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.019458    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-tmp-dir\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:26:30.019500 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.019479    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-home\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:26:30.019500 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.019493    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-model-cache\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:26:30.021223 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.021194    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-dshm" (OuterVolumeSpecName: "dshm") pod "ed4bf1dd-8dc5-4d98-9596-57abd80b707f" (UID: "ed4bf1dd-8dc5-4d98-9596-57abd80b707f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:26:30.021354 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.021218    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed4bf1dd-8dc5-4d98-9596-57abd80b707f" (UID: "ed4bf1dd-8dc5-4d98-9596-57abd80b707f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:26:30.021354 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.021282    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-kube-api-access-mt69w" (OuterVolumeSpecName: "kube-api-access-mt69w") pod "ed4bf1dd-8dc5-4d98-9596-57abd80b707f" (UID: "ed4bf1dd-8dc5-4d98-9596-57abd80b707f"). InnerVolumeSpecName "kube-api-access-mt69w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:26:30.120292 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.120223    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-tls-certs\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:26:30.120292 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.120248    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt69w\" (UniqueName: \"kubernetes.io/projected/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-kube-api-access-mt69w\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:26:30.120292 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.120257    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4bf1dd-8dc5-4d98-9596-57abd80b707f-dshm\") on node \"ip-10-0-143-250.ec2.internal\" DevicePath \"\""
Sep 14 08:26:30.570153 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.570059    2563 generic.go:356] "Generic (PLEG): container finished" podID="ed4bf1dd-8dc5-4d98-9596-57abd80b707f" containerID="c8915a2528849cfb5a6d398f62b7474877bd81a4500a25ba8f82687c526cb4d0" exitCode=0
Sep 14 08:26:30.570153 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.570108    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"
Sep 14 08:26:30.570374 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.570149    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw" event={"ID":"ed4bf1dd-8dc5-4d98-9596-57abd80b707f","Type":"ContainerDied","Data":"c8915a2528849cfb5a6d398f62b7474877bd81a4500a25ba8f82687c526cb4d0"}
Sep 14 08:26:30.570374 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.570192    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw" event={"ID":"ed4bf1dd-8dc5-4d98-9596-57abd80b707f","Type":"ContainerDied","Data":"57a859bb2d22cffe00e39bb01a20c6bd41647d1e53975f0562f2ac592d892d97"}
Sep 14 08:26:30.570374 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.570218    2563 scope.go:117] "RemoveContainer" containerID="c8915a2528849cfb5a6d398f62b7474877bd81a4500a25ba8f82687c526cb4d0"
Sep 14 08:26:30.579770 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.579753    2563 scope.go:117] "RemoveContainer" containerID="c8915a2528849cfb5a6d398f62b7474877bd81a4500a25ba8f82687c526cb4d0"
Sep 14 08:26:30.580056 ip-10-0-143-250 kubenswrapper[2563]: E0914 08:26:30.580038    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8915a2528849cfb5a6d398f62b7474877bd81a4500a25ba8f82687c526cb4d0\": container with ID starting with c8915a2528849cfb5a6d398f62b7474877bd81a4500a25ba8f82687c526cb4d0 not found: ID does not exist" containerID="c8915a2528849cfb5a6d398f62b7474877bd81a4500a25ba8f82687c526cb4d0"
Sep 14 08:26:30.580116 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.580065    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8915a2528849cfb5a6d398f62b7474877bd81a4500a25ba8f82687c526cb4d0"} err="failed to get container status \"c8915a2528849cfb5a6d398f62b7474877bd81a4500a25ba8f82687c526cb4d0\": rpc error: code = NotFound desc = could not find container \"c8915a2528849cfb5a6d398f62b7474877bd81a4500a25ba8f82687c526cb4d0\": container with ID starting with c8915a2528849cfb5a6d398f62b7474877bd81a4500a25ba8f82687c526cb4d0 not found: ID does not exist"
Sep 14 08:26:30.593125 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.593098    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"]
Sep 14 08:26:30.600241 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:30.600219    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-96ce3f1c-kserve-7cbfb74f79-dlxhw"]
Sep 14 08:26:31.885824 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:26:31.885795    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed4bf1dd-8dc5-4d98-9596-57abd80b707f" path="/var/lib/kubelet/pods/ed4bf1dd-8dc5-4d98-9596-57abd80b707f/volumes"
Sep 14 08:27:00.423343 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:00.423258    2563 scope.go:117] "RemoveContainer" containerID="c9ecda32d63227b0765c0a40a22b07ca08ffbb687d0efe3985af2e3c52cbaa41"
Sep 14 08:27:00.431956 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:00.431935    2563 scope.go:117] "RemoveContainer" containerID="1195d60da1378f8ca2d9d217c50a9a220a3515dd0451dd1ba422b3850eb48f1b"
Sep 14 08:27:07.658972 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:07.658923    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6f9xh_a3ddd337-89aa-407a-ac0a-8d503cc3ef96/manager/0.log"
Sep 14 08:27:07.787024 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:07.786988    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6945fd44b-hndfp_babff1f3-9f42-40f7-8112-3c9be4f3e13b/manager/0.log"
Sep 14 08:27:08.270784 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:08.270756    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c4c5b687-nvmjc_2a4a1d7c-fb45-43e7-a90e-3d300bee86a1/copy-manifests/0.log"
Sep 14 08:27:08.278601 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:08.278577    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c4c5b687-nvmjc_2a4a1d7c-fb45-43e7-a90e-3d300bee86a1/manager/0.log"
Sep 14 08:27:08.964176 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:08.964143    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29822880-hllm7_68393035-a1a6-462e-b4a1-935a74ac0d71/cleanup/0.log"
Sep 14 08:27:09.189479 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:09.189449    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-bkpkw_02642698-3e24-4da5-9b12-6aa77b31f925/postgres/0.log"
Sep 14 08:27:10.447396 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:10.447365    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-25nq6_1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9/manager/0.log"
Sep 14 08:27:10.682466 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:10.682437    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-hk89x_43ebfe24-78eb-4c7d-a77c-4ea96d27269a/manager/0.log"
Sep 14 08:27:11.162949 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:11.162922    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-r2gfm_caea3af0-2fe0-4035-8098-21118882dff1/discovery/0.log"
Sep 14 08:27:11.388457 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:11.388430    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86f4846dcd-tnz86_64cd7e59-9eb7-48f2-bbcd-d12df7334aca/kube-auth-proxy/0.log"
Sep 14 08:27:11.743993 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:11.743960    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jhhdc_78828de1-4078-4002-a86b-da44a9b7120f/payload-processing/0.log"
Sep 14 08:27:11.859305 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:11.859276    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68c9854f4b-68nfz_01a4583c-ddea-4d2e-8d11-61ebef63ef24/router/0.log"
Sep 14 08:27:12.206256 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:12.206225    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-h5j65_6eb6f09f-01ee-44a0-961b-e288d02c820c/main/0.log"
Sep 14 08:27:12.322174 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:12.322146    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vbftw_1aee68db-f5a3-48e1-b25e-d5ff2cf38833/main/0.log"
Sep 14 08:27:22.143503 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:22.143469    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zwmpr_bee2d6cd-aca8-4008-be09-34cd9916d966/global-pull-secret-syncer/0.log"
Sep 14 08:27:22.334436 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:22.334401    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ffvbt_61f94071-3e09-4079-abb1-bbfa74ae8453/konnectivity-agent/0.log"
Sep 14 08:27:22.587636 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:22.587611    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-250.ec2.internal_520462c720d404adb820140db50e2930/haproxy/0.log"
Sep 14 08:27:27.328123 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:27.328096    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-25nq6_1fdf9b58-e57a-4bc0-b1c0-51a5239c73e9/manager/0.log"
Sep 14 08:27:27.451270 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:27.451242    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-hk89x_43ebfe24-78eb-4c7d-a77c-4ea96d27269a/manager/0.log"
Sep 14 08:27:29.032609 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:29.032568    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-hj5zz_6ba25a8b-f03d-423f-b432-550618dfd5b4/cluster-monitoring-operator/0.log"
Sep 14 08:27:29.057504 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:29.057474    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2gs86_04790d5e-1c25-4b84-8e27-b4655f6e50ee/kube-state-metrics/0.log"
Sep 14 08:27:29.082313 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:29.082290    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2gs86_04790d5e-1c25-4b84-8e27-b4655f6e50ee/kube-rbac-proxy-main/0.log"
Sep 14 08:27:29.104291 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:29.104272    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-2gs86_04790d5e-1c25-4b84-8e27-b4655f6e50ee/kube-rbac-proxy-self/0.log"
Sep 14 08:27:29.136993 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:29.136971    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5b6cdf8787-kxws6_a4d24038-849b-4139-8f73-730159f4421e/metrics-server/0.log"
Sep 14 08:27:29.183357 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:29.183333    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-kvjf9_172d736a-dd5b-47a8-b6f1-ea0b4c0af7af/monitoring-plugin/0.log"
Sep 14 08:27:29.214098 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:29.214056    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8hbxw_4034874d-df6a-4e24-a248-fe735722dc7b/node-exporter/0.log"
Sep 14 08:27:29.239728 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:29.239692    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8hbxw_4034874d-df6a-4e24-a248-fe735722dc7b/kube-rbac-proxy/0.log"
Sep 14 08:27:29.262855 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:29.262836    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8hbxw_4034874d-df6a-4e24-a248-fe735722dc7b/init-textfile/0.log"
Sep 14 08:27:29.449932 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:29.449910    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-9wdh8_39196832-8ece-4519-b222-0f5d5043422a/kube-rbac-proxy-main/0.log"
Sep 14 08:27:29.471049 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:29.471029    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-9wdh8_39196832-8ece-4519-b222-0f5d5043422a/kube-rbac-proxy-self/0.log"
Sep 14 08:27:29.492593 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:29.492570    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-9wdh8_39196832-8ece-4519-b222-0f5d5043422a/openshift-state-metrics/0.log"
Sep 14 08:27:31.898862 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:31.898830    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/1.log"
Sep 14 08:27:31.904803 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:31.904777    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-jj5qw_af87c4a9-ecde-4180-bba1-da2834189fb4/console-operator/2.log"
Sep 14 08:27:31.993755 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:31.993725    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"]
Sep 14 08:27:31.994115 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:31.994103    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed4bf1dd-8dc5-4d98-9596-57abd80b707f" containerName="main"
Sep 14 08:27:31.994164 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:31.994117    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed4bf1dd-8dc5-4d98-9596-57abd80b707f" containerName="main"
Sep 14 08:27:31.994199 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:31.994173    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed4bf1dd-8dc5-4d98-9596-57abd80b707f" containerName="main"
Sep 14 08:27:31.997175 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:31.997156    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:31.999804 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:31.999783    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ds52d\"/\"default-dockercfg-s6bnm\""
Sep 14 08:27:31.999918 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:31.999860    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ds52d\"/\"kube-root-ca.crt\""
Sep 14 08:27:31.999918 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:31.999904    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ds52d\"/\"openshift-service-ca.crt\""
Sep 14 08:27:32.006887 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.006866    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"]
Sep 14 08:27:32.176142 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.176055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2dd19d03-9f41-42d3-97e6-62b0ff146222-sys\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.176142 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.176097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2dd19d03-9f41-42d3-97e6-62b0ff146222-proc\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.176331 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.176160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpw9m\" (UniqueName: \"kubernetes.io/projected/2dd19d03-9f41-42d3-97e6-62b0ff146222-kube-api-access-mpw9m\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.176331 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.176232    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2dd19d03-9f41-42d3-97e6-62b0ff146222-podres\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.176331 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.176293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2dd19d03-9f41-42d3-97e6-62b0ff146222-lib-modules\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.277270 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.277243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2dd19d03-9f41-42d3-97e6-62b0ff146222-sys\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.277429 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.277279    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2dd19d03-9f41-42d3-97e6-62b0ff146222-proc\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.277429 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.277300    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpw9m\" (UniqueName: \"kubernetes.io/projected/2dd19d03-9f41-42d3-97e6-62b0ff146222-kube-api-access-mpw9m\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.277429 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.277342    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2dd19d03-9f41-42d3-97e6-62b0ff146222-podres\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.277429 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.277362    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2dd19d03-9f41-42d3-97e6-62b0ff146222-lib-modules\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.277429 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.277376    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2dd19d03-9f41-42d3-97e6-62b0ff146222-proc\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.277429 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.277376    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2dd19d03-9f41-42d3-97e6-62b0ff146222-sys\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.277692 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.277503    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2dd19d03-9f41-42d3-97e6-62b0ff146222-podres\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.277692 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.277509    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2dd19d03-9f41-42d3-97e6-62b0ff146222-lib-modules\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.285691 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.285668    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpw9m\" (UniqueName: \"kubernetes.io/projected/2dd19d03-9f41-42d3-97e6-62b0ff146222-kube-api-access-mpw9m\") pod \"perf-node-gather-daemonset-wmnl4\" (UID: \"2dd19d03-9f41-42d3-97e6-62b0ff146222\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.307545 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.307526    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.477245 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.477176    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7679fc7b7c-hhqbr_b959c0f7-7a0d-434a-805a-9bc9e8075fd3/console/0.log"
Sep 14 08:27:32.574067 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.574041    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"]
Sep 14 08:27:32.575112 ip-10-0-143-250 kubenswrapper[2563]: W0914 08:27:32.575088    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2dd19d03_9f41_42d3_97e6_62b0ff146222.slice/crio-40c7ee865975dfb0071d20979ed9a37de391bb8c34a986c2ac329fac40b8e1b6 WatchSource:0}: Error finding container 40c7ee865975dfb0071d20979ed9a37de391bb8c34a986c2ac329fac40b8e1b6: Status 404 returned error can't find the container with id 40c7ee865975dfb0071d20979ed9a37de391bb8c34a986c2ac329fac40b8e1b6
Sep 14 08:27:32.917838 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.917803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4" event={"ID":"2dd19d03-9f41-42d3-97e6-62b0ff146222","Type":"ContainerStarted","Data":"e5fc173719724c65abcbf02f15c87bb77967476b51858ca534dd42fd6562dee7"}
Sep 14 08:27:32.918225 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.917856    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4" event={"ID":"2dd19d03-9f41-42d3-97e6-62b0ff146222","Type":"ContainerStarted","Data":"40c7ee865975dfb0071d20979ed9a37de391bb8c34a986c2ac329fac40b8e1b6"}
Sep 14 08:27:32.918225 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.918005    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:32.937052 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:32.936995    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4" podStartSLOduration=1.9369752070000001 podStartE2EDuration="1.936975207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 08:27:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:27:32.934357868 +0000 UTC m=+2373.563245515" watchObservedRunningTime="2026-09-14 08:27:32.936975207 +0000 UTC m=+2373.565862835"
Sep 14 08:27:33.193555 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:33.193478    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-c29vb_73263911-8bba-4703-ac5f-8b7bc2913a68/volume-data-source-validator/0.log"
Sep 14 08:27:34.515606 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:34.515568    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mk8vk_9a90a408-7211-425a-9227-84e964f6d045/dns/0.log"
Sep 14 08:27:34.548091 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:34.548066    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mk8vk_9a90a408-7211-425a-9227-84e964f6d045/kube-rbac-proxy/0.log"
Sep 14 08:27:34.632473 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:34.632449    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ts5z9_261155e3-6b52-449b-94ca-649f7210ccd9/dns-node-resolver/0.log"
Sep 14 08:27:35.204117 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:35.204084    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-775bff4dfb-tc9bw_8c625d4c-9cbb-4930-93e1-8b771503e661/registry/0.log"
Sep 14 08:27:35.290601 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:35.290574    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r4h2b_416afa6c-5c55-487f-8de6-200bc0be3e06/node-ca/0.log"
Sep 14 08:27:36.666513 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:36.666480    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-r2gfm_caea3af0-2fe0-4035-8098-21118882dff1/discovery/0.log"
Sep 14 08:27:36.798886 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:36.798859    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86f4846dcd-tnz86_64cd7e59-9eb7-48f2-bbcd-d12df7334aca/kube-auth-proxy/0.log"
Sep 14 08:27:36.900375 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:36.900346    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jhhdc_78828de1-4078-4002-a86b-da44a9b7120f/payload-processing/0.log"
Sep 14 08:27:36.922339 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:36.922273    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68c9854f4b-68nfz_01a4583c-ddea-4d2e-8d11-61ebef63ef24/router/0.log"
Sep 14 08:27:37.863829 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:37.863802    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s6ld9_bbc0569a-8541-4a2a-b92c-80292eff74b5/serve-healthcheck-canary/0.log"
Sep 14 08:27:38.403764 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:38.403736    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vmctf_28a1ffa9-6f79-48eb-b760-bd9833f20c45/kube-rbac-proxy/0.log"
Sep 14 08:27:38.431534 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:38.431512    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vmctf_28a1ffa9-6f79-48eb-b760-bd9833f20c45/exporter/0.log"
Sep 14 08:27:38.456438 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:38.456420    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vmctf_28a1ffa9-6f79-48eb-b760-bd9833f20c45/extractor/0.log"
Sep 14 08:27:38.932858 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:38.932833    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-wmnl4"
Sep 14 08:27:40.852875 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:40.852829    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6f9xh_a3ddd337-89aa-407a-ac0a-8d503cc3ef96/manager/0.log"
Sep 14 08:27:40.908348 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:40.908317    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6945fd44b-hndfp_babff1f3-9f42-40f7-8112-3c9be4f3e13b/manager/0.log"
Sep 14 08:27:41.177545 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:41.177463    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c4c5b687-nvmjc_2a4a1d7c-fb45-43e7-a90e-3d300bee86a1/manager/0.log"
Sep 14 08:27:41.201938 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:41.201913    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c4c5b687-nvmjc_2a4a1d7c-fb45-43e7-a90e-3d300bee86a1/copy-manifests/0.log"
Sep 14 08:27:42.430794 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:42.430763    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-r2gfm_caea3af0-2fe0-4035-8098-21118882dff1/discovery/0.log"
Sep 14 08:27:42.460168 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:42.460141    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86f4846dcd-tnz86_64cd7e59-9eb7-48f2-bbcd-d12df7334aca/kube-auth-proxy/0.log"
Sep 14 08:27:42.538392 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:42.538361    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jhhdc_78828de1-4078-4002-a86b-da44a9b7120f/payload-processing/0.log"
Sep 14 08:27:42.551564 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:42.551537    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68c9854f4b-68nfz_01a4583c-ddea-4d2e-8d11-61ebef63ef24/router/0.log"
Sep 14 08:27:42.744663 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:42.744620    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-vmkvh_b6351179-d647-424e-a7ba-87389b3df3fe/openshift-lws-operator/0.log"
Sep 14 08:27:44.518892 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:44.518798    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-r2gfm_caea3af0-2fe0-4035-8098-21118882dff1/discovery/0.log"
Sep 14 08:27:44.556604 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:44.556581    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86f4846dcd-tnz86_64cd7e59-9eb7-48f2-bbcd-d12df7334aca/kube-auth-proxy/0.log"
Sep 14 08:27:44.634947 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:44.634920    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-jhhdc_78828de1-4078-4002-a86b-da44a9b7120f/payload-processing/0.log"
Sep 14 08:27:44.652396 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:44.652374    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68c9854f4b-68nfz_01a4583c-ddea-4d2e-8d11-61ebef63ef24/router/0.log"
Sep 14 08:27:48.284375 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:48.284343    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-hb2jh_4e45fd58-3953-4aca-89d6-d8facaaf9a28/kube-storage-version-migrator-operator/1.log"
Sep 14 08:27:48.285185 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:48.285166    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-hb2jh_4e45fd58-3953-4aca-89d6-d8facaaf9a28/kube-storage-version-migrator-operator/0.log"
Sep 14 08:27:49.351653 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:49.351615    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7z55c_ef0aa66b-f097-4d15-b247-93ec7a1bdb3c/kube-multus/0.log"
Sep 14 08:27:49.694372 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:49.694302    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pk_d9f00ee5-9070-420e-9fe1-3e5e0482610e/kube-multus-additional-cni-plugins/0.log"
Sep 14 08:27:49.716586 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:49.716564    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pk_d9f00ee5-9070-420e-9fe1-3e5e0482610e/egress-router-binary-copy/0.log"
Sep 14 08:27:49.736516 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:49.736494    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pk_d9f00ee5-9070-420e-9fe1-3e5e0482610e/cni-plugins/0.log"
Sep 14 08:27:49.754922 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:49.754905    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pk_d9f00ee5-9070-420e-9fe1-3e5e0482610e/bond-cni-plugin/0.log"
Sep 14 08:27:49.774786 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:49.774766    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pk_d9f00ee5-9070-420e-9fe1-3e5e0482610e/routeoverride-cni/0.log"
Sep 14 08:27:49.796534 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:49.796517    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pk_d9f00ee5-9070-420e-9fe1-3e5e0482610e/whereabouts-cni-bincopy/0.log"
Sep 14 08:27:49.815526 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:49.815505    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v22pk_d9f00ee5-9070-420e-9fe1-3e5e0482610e/whereabouts-cni/0.log"
Sep 14 08:27:49.892402 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:49.892376    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-46qgp_c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257/network-metrics-daemon/0.log"
Sep 14 08:27:49.910288 ip-10-0-143-250 kubenswrapper[2563]: I0914 08:27:49.910266    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-46qgp_c1e3aa91-6cdf-4878-a9c1-12f7e5dfd257/kube-rbac-proxy/0.log"