May 25 13:05:33.153996 ip-10-0-129-151 systemd[1]: Starting Kubernetes Kubelet...
May 25 13:05:33.614833 ip-10-0-129-151 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 25 13:05:33.614833 ip-10-0-129-151 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 25 13:05:33.614833 ip-10-0-129-151 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 25 13:05:33.614833 ip-10-0-129-151 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 25 13:05:33.614833 ip-10-0-129-151 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 25 13:05:33.617561 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.617471    2559 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 25 13:05:33.621231 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621213    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 13:05:33.621231 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621229    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 13:05:33.621231 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621236    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621241    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621246    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621250    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621254    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621258    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621263    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621267    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621272    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621276    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621280    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621284    2559 feature_gate.go:328] unrecognized feature gate: Example
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621287    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621292    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621296    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621300    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621304    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621308    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621312    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 13:05:33.621419 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621326    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621330    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621334    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621337    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621342    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621347    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621351    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621355    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621359    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621363    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621367    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621371    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621375    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621379    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621385    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621390    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621395    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621399    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621403    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 13:05:33.622210 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621407    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621414    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621420    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621425    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621429    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621433    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621437    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621442    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621446    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621450    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621454    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621459    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621464    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621469    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621474    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621478    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621484    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621490    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621494    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 13:05:33.622972 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621499    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621503    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621508    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621512    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621516    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621520    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621524    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621528    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621533    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621539    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621543    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621549    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621553    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621558    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621564    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621568    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621573    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621577    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621581    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621586    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 25 13:05:33.623448 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621590    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621594    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621598    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621602    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621606    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621610    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.621615    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622268    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622277    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622282    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622286    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622291    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622295    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622301    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622305    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622309    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622313    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622317    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622321    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622326    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 13:05:33.623942 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622330    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622335    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622340    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622345    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622349    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622355    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622359    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622364    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622368    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622372    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622376    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622380    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622384    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622389    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622392    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622397    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622401    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622405    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622409    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622414    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 13:05:33.625329 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622418    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622422    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622426    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622430    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622434    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622438    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622444    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622448    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622452    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622456    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622460    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622465    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622469    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622474    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622478    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622483    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622487    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622492    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622496    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 13:05:33.625863 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622500    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622504    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622508    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622512    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622516    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622520    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622524    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622528    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622532    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622536    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622541    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622545    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622549    2559 feature_gate.go:328] unrecognized feature gate: Example
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622553    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622557    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622561    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622566    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622573    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622578    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 13:05:33.626442 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622582    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622587    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622592    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622596    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622602    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622607    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622614    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622619    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622623    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622628    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622632    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622637    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622642    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622646    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.622650    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624062    2559 flags.go:64] FLAG: --address="0.0.0.0"
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624077    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624089    2559 flags.go:64] FLAG: --anonymous-auth="true"
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624097    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624104    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624109    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 25 13:05:33.626976 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624116    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624123    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624129    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624134    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624140    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624145    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624150    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624155    2559 flags.go:64] FLAG: --cgroup-root=""
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624160    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624165    2559 flags.go:64] FLAG: --client-ca-file=""
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624169    2559 flags.go:64] FLAG: --cloud-config=""
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624174    2559 flags.go:64] FLAG: --cloud-provider="external"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624179    2559 flags.go:64] FLAG: --cluster-dns="[]"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624186    2559 flags.go:64] FLAG: --cluster-domain=""
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624191    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624198    2559 flags.go:64] FLAG: --config-dir=""
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624202    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624208    2559 flags.go:64] FLAG: --container-log-max-files="5"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624214    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624219    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624225    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624230    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624235    2559 flags.go:64] FLAG: --contention-profiling="false"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624242    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 25 13:05:33.627575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624247    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624252    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624257    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624264    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624269    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624274    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624279    2559 flags.go:64] FLAG: --enable-load-reader="false"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624284    2559 flags.go:64] FLAG: --enable-server="true"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624289    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624297    2559 flags.go:64] FLAG: --event-burst="100"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624302    2559 flags.go:64] FLAG: --event-qps="50"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624307    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624313    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624318    2559 flags.go:64] FLAG: --eviction-hard=""
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624324    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624328    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624333    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624339    2559 flags.go:64] FLAG: --eviction-soft=""
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624343    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624348    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624353    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624359    2559 flags.go:64] FLAG: --experimental-mounter-path=""
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624364    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624369    2559 flags.go:64] FLAG: --fail-swap-on="true"
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624374    2559 flags.go:64] FLAG: --feature-gates=""
May 25 13:05:33.628179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624381    2559 flags.go:64] FLAG: --file-check-frequency="20s"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624386    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624391    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624397    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624402    2559 flags.go:64] FLAG: --healthz-port="10248"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624407    2559 flags.go:64] FLAG: --help="false"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624413    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-129-151.ec2.internal"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624419    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624424    2559 flags.go:64] FLAG: --http-check-frequency="20s"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624429    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624434    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624440    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624445    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624450    2559 flags.go:64] FLAG: --image-service-endpoint=""
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624454    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624459    2559 flags.go:64] FLAG: --kube-api-burst="100"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624464    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624470    2559 flags.go:64] FLAG: --kube-api-qps="50"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624474    2559 flags.go:64] FLAG: --kube-reserved=""
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624479    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624484    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624489    2559 flags.go:64] FLAG: --kubelet-cgroups=""
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624494    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624499    2559 flags.go:64] FLAG: --lock-file=""
May 25 13:05:33.628796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624503    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624513    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624518    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624527    2559 flags.go:64] FLAG: --log-json-split-stream="false"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624532    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624538    2559 flags.go:64] FLAG: --log-text-split-stream="false"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624543    2559 flags.go:64] FLAG: --logging-format="text"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624548    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624553    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624558    2559 flags.go:64] FLAG: --manifest-url=""
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624562    2559 flags.go:64] FLAG: --manifest-url-header=""
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624569    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624574    2559 flags.go:64] FLAG: --max-open-files="1000000"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624579    2559 flags.go:64] FLAG: --max-pods="110"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624582    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624587    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624590    2559 flags.go:64] FLAG: --memory-manager-policy="None"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624593    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624596    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624599    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624602    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624610    2559 flags.go:64] FLAG: --node-status-max-images="50"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624613    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624616    2559 flags.go:64] FLAG: --oom-score-adj="-999"
May 25 13:05:33.629388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624619    2559 flags.go:64] FLAG: --pod-cidr=""
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624622    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624628    2559 flags.go:64] FLAG: --pod-manifest-path=""
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624631    2559 flags.go:64] FLAG: --pod-max-pids="-1"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624635    2559 flags.go:64] FLAG: --pods-per-core="0"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624638    2559 flags.go:64] FLAG: --port="10250"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624640    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624643    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-009b9c774ec0edcea"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624646    2559 flags.go:64] FLAG: --qos-reserved=""
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624650    2559 flags.go:64] FLAG: --read-only-port="10255"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624655    2559 flags.go:64] FLAG: --register-node="true"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624658    2559 flags.go:64] FLAG: --register-schedulable="true"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624661    2559 flags.go:64] FLAG: --register-with-taints=""
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624665    2559 flags.go:64] FLAG: --registry-burst="10"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624668    2559 flags.go:64] FLAG: --registry-qps="5"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624670    2559 flags.go:64] FLAG: --reserved-cpus=""
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624674    2559 flags.go:64] FLAG: --reserved-memory=""
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624678    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624681    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624684    2559 flags.go:64] FLAG: --rotate-certificates="false"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624687    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624689    2559 flags.go:64] FLAG: --runonce="false"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624693    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624696    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624699    2559 flags.go:64] FLAG: --seccomp-default="false"
May 25 13:05:33.629995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624702    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624705    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624708    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624711    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624731    2559 flags.go:64] FLAG: --storage-driver-password="root"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624736    2559 flags.go:64] FLAG: --storage-driver-secure="false"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624739    2559 flags.go:64] FLAG: --storage-driver-table="stats"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624742    2559 flags.go:64] FLAG: --storage-driver-user="root"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624745    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624748    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624751    2559 flags.go:64] FLAG: --system-cgroups=""
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624754    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624760    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624763    2559 flags.go:64] FLAG: --tls-cert-file=""
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624766    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624770    2559 flags.go:64] FLAG: --tls-min-version=""
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624773    2559 flags.go:64] FLAG: --tls-private-key-file=""
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624777    2559 flags.go:64] FLAG: --topology-manager-policy="none"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624780    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624787    2559 flags.go:64] FLAG: --topology-manager-scope="container"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624791    2559 flags.go:64] FLAG: --v="2"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624795    2559 flags.go:64] FLAG: --version="false"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624800    2559 flags.go:64] FLAG: --vmodule=""
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624804    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.624807    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 25 13:05:33.630598 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624914    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624917    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624920    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624923    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624925    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624928    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624930    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624933    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624936    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624939    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624942    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624945    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624947    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624950    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624953    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624955    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624958    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624960    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624963    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624966    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 13:05:33.631251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624968    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624971    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624973    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624976    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624979    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624982    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624986    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624989    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624992    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624994    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624997    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.624999    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625002    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625005    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625007    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625009    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625012    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625015    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625017    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 13:05:33.631809 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625020    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625022    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625026    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625030    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625034    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625036    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625039    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625042    2559 feature_gate.go:328] unrecognized feature gate: Example
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625045    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625048    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625050    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625053    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625055    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625058    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625062    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625065    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625069    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625072    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625075    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 13:05:33.632279 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625079    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625082    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625085    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625087    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625089    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625092    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625095    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625097    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625099    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625102    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625105    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625107    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625110    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625113    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625115    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625118    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625121    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625124    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625126    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625128    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 13:05:33.632769 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625131    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 13:05:33.633250 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625133    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 13:05:33.633250 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625136    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 13:05:33.633250 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625138    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 13:05:33.633250 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625141    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 13:05:33.633250 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625144    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 13:05:33.633250 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625146    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 13:05:33.633250 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.625149    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 13:05:33.633250 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.625991    2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.633262    2559 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.633280    2559 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633328    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633332    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633336    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633340    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633343    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633346    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633349    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633352    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633354    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633357    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633360    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633362    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633365    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633368    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633370    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633373    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 13:05:33.633453 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633376    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633380    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633383    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633386    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633389    2559 feature_gate.go:328] unrecognized feature gate: Example
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633392    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633394    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633397    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633399    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633402    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633404    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633407    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633410    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633412    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633415    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633418    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633421    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633424    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633428    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 13:05:33.633967 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633434    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633438    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633442    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633445    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633448    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633451    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633453    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633456    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633459    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633463    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633466    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633469    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633472    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633474    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633477    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633480    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633483    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633485    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633488    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633491    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 13:05:33.634451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633494    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633496    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633499    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633501    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633504    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633506    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633510    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633512    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633515    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633517    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633520    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633522    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633525    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633527    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633530    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633532    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633535    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633537    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633540    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633542    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 13:05:33.634958 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633545    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633547    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633550    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633552    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633555    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633557    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633560    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633562    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633565    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633568    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633570    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.633576    2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633671    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633676    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633679    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 13:05:33.635476 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633682    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633685    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633687    2559 feature_gate.go:328] unrecognized feature gate: Example2
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633690    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633693    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633696    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633699    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633702    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633704    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633707    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633710    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633712    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633715    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633732    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633734    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633737    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633740    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633742    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633745    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633748    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 13:05:33.635865 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633750    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633753    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633756    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633760    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633764    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633767    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633770    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633773    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633783    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633787    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633791    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633793    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633796    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633799    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633801    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633804    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633807    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633809    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633812    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 13:05:33.636336 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633814    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633817    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633821    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633824    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633826    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633829    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633831    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633834    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633836    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633839    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633842    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633844    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633847    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633849    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633852    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633854    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633857    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633859    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633862    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633864    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 13:05:33.636860 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633867    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633870    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633872    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633875    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633878    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633880    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633882    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633885    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633887    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633890    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633893    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633895    2559 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633898    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633900    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633903    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633905    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633908    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633912    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633915    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 13:05:33.637355 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633918    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 13:05:33.637930 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633921    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 13:05:33.637930 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633924    2559 feature_gate.go:328] unrecognized feature gate: Example
May 25 13:05:33.637930 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633926    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 13:05:33.637930 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:33.633929    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 13:05:33.637930 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.633934    2559 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 25 13:05:33.637930 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.634634    2559 server.go:962] "Client rotation is on, will bootstrap in background"
May 25 13:05:33.637930 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.636600    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 25 13:05:33.637930 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.637679    2559 server.go:1019] "Starting client certificate rotation"
May 25 13:05:33.638157 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.638139    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 25 13:05:33.638801 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.638788    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 25 13:05:33.663693 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.663673    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 25 13:05:33.668942 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.668913    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 25 13:05:33.681311 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.681288    2559 log.go:25] "Validated CRI v1 runtime API"
May 25 13:05:33.686779 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.686759    2559 log.go:25] "Validated CRI v1 image API"
May 25 13:05:33.689007 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.688989    2559 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 25 13:05:33.694240 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.694221    2559 fs.go:135] Filesystem UUIDs: map[778f6484-82c3-4cc8-a36f-839da7aa7a04:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 967b68b5-ab15-4fe3-8f3f-a0d4b5340302:/dev/nvme0n1p4]
May 25 13:05:33.694310 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.694240    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 25 13:05:33.698552 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.698442    2559 manager.go:217] Machine: {Timestamp:2026-05-25 13:05:33.697930136 +0000 UTC m=+0.416893265 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099621 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23fd494b52247cc9dae1065e8f7e5b SystemUUID:ec23fd49-4b52-247c-c9da-e1065e8f7e5b BootID:114c3981-fda6-4702-bbc2-c15249152521 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:85:b4:24:7e:77 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:85:b4:24:7e:77 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7a:25:a0:a6:ac:2b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 25 13:05:33.698552 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.698550    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 25 13:05:33.698658 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.698630    2559 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 25 13:05:33.699666 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.699643    2559 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 25 13:05:33.699835 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.699669    2559 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-151.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 25 13:05:33.699878 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.699847    2559 topology_manager.go:138] "Creating topology manager with none policy"
May 25 13:05:33.699878 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.699856    2559 container_manager_linux.go:306] "Creating device plugin manager"
May 25 13:05:33.700481 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.700471    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 25 13:05:33.701245 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.701235    2559 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 25 13:05:33.702960 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.702948    2559 state_mem.go:36] "Initialized new in-memory state store"
May 25 13:05:33.703080 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.703072    2559 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 25 13:05:33.706749 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.706737    2559 kubelet.go:491] "Attempting to sync node with API server"
May 25 13:05:33.706799 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.706756    2559 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 25 13:05:33.706799 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.706771    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 25 13:05:33.706799 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.706780    2559 kubelet.go:397] "Adding apiserver pod source"
May 25 13:05:33.706799 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.706790    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 25 13:05:33.708288 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.708275    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 25 13:05:33.708343 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.708295    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 25 13:05:33.710167 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.710134    2559 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 25 13:05:33.712101 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.712086    2559 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 25 13:05:33.715255 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.715234    2559 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 25 13:05:33.716112 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.716087    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 25 13:05:33.716206 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.716122    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 25 13:05:33.716206 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.716133    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 25 13:05:33.716206 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.716142    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 25 13:05:33.716206 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.716155    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 25 13:05:33.716206 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.716165    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 25 13:05:33.716206 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.716174    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 25 13:05:33.716206 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.716183    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 25 13:05:33.716206 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.716194    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 25 13:05:33.716206 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.716203    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 25 13:05:33.716489 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.716221    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 25 13:05:33.717629 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.717613    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 25 13:05:33.717760 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.717748    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 25 13:05:33.717826 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.717765    2559 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 25 13:05:33.721437 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.721421    2559 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 25 13:05:33.721507 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.721469    2559 server.go:1295] "Started kubelet"
May 25 13:05:33.721603 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.721555    2559 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 25 13:05:33.721640 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.721624    2559 server_v1.go:47] "podresources" method="list" useActivePods=true
May 25 13:05:33.722159 ip-10-0-129-151 systemd[1]: Started Kubernetes Kubelet.
May 25 13:05:33.722841 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.722825    2559 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 25 13:05:33.722898 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.722825    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 25 13:05:33.723463 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.723448    2559 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-151.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 25 13:05:33.724244 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.724222    2559 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 25 13:05:33.724294 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.724225    2559 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-151.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 25 13:05:33.725850 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.725831    2559 server.go:317] "Adding debug handlers to kubelet server"
May 25 13:05:33.730323 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.730303    2559 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 25 13:05:33.730664 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.730648    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 25 13:05:33.731130 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.731104    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 25 13:05:33.731342 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.731327    2559 volume_manager.go:295] "The desired_state_of_world populator starts"
May 25 13:05:33.731430 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.731343    2559 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 25 13:05:33.731485 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.731468    2559 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 25 13:05:33.731538 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.731532    2559 reconstruct.go:97] "Volume reconstruction finished"
May 25 13:05:33.731583 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.731540    2559 reconciler.go:26] "Reconciler: start to sync state"
May 25 13:05:33.732413 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.732395    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 25 13:05:33.732501 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.732435    2559 factory.go:55] Registering systemd factory
May 25 13:05:33.732501 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.732448    2559 factory.go:223] Registration of the systemd container factory successfully
May 25 13:05:33.732602 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.732580    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-151.ec2.internal\" not found"
May 25 13:05:33.733201 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.733162    2559 factory.go:153] Registering CRI-O factory
May 25 13:05:33.733201 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.733179    2559 factory.go:223] Registration of the crio container factory successfully
May 25 13:05:33.733201 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.733199    2559 factory.go:103] Registering Raw factory
May 25 13:05:33.733382 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.733212    2559 manager.go:1196] Started watching for new ooms in manager
May 25 13:05:33.733591 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.733564    2559 manager.go:319] Starting recovery of all containers
May 25 13:05:33.738177 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.738148    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-151.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 25 13:05:33.738660 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.738507    2559 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 25 13:05:33.739265 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.738403    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-151.ec2.internal.18b2d0c1d7231bb3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-151.ec2.internal,UID:ip-10-0-129-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-151.ec2.internal,},FirstTimestamp:2026-05-25 13:05:33.721435059 +0000 UTC m=+0.440398189,LastTimestamp:2026-05-25 13:05:33.721435059 +0000 UTC m=+0.440398189,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-151.ec2.internal,}"
May 25 13:05:33.743331 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.743309    2559 manager.go:324] Recovery completed
May 25 13:05:33.748511 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.748497    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 13:05:33.750309 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.750293    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasSufficientMemory"
May 25 13:05:33.750359 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.750322    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasNoDiskPressure"
May 25 13:05:33.750359 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.750332    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasSufficientPID"
May 25 13:05:33.750869 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.750854    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 25 13:05:33.750869 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.750868    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 25 13:05:33.750956 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.750883    2559 state_mem.go:36] "Initialized new in-memory state store"
May 25 13:05:33.752189 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.752128    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-151.ec2.internal.18b2d0c1d8dbb3c5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-151.ec2.internal,UID:ip-10-0-129-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-151.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-151.ec2.internal,},FirstTimestamp:2026-05-25 13:05:33.750309829 +0000 UTC m=+0.469272961,LastTimestamp:2026-05-25 13:05:33.750309829 +0000 UTC m=+0.469272961,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-151.ec2.internal,}"
May 25 13:05:33.753266 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.753253    2559 policy_none.go:49] "None policy: Start"
May 25 13:05:33.753317 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.753270    2559 memory_manager.go:186] "Starting memorymanager" policy="None"
May 25 13:05:33.753317 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.753282    2559 state_mem.go:35] "Initializing new in-memory state store"
May 25 13:05:33.757714 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.757699    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qnlb2"
May 25 13:05:33.764487 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.764427    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-151.ec2.internal.18b2d0c1d8dbf7c7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-151.ec2.internal,UID:ip-10-0-129-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-151.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-151.ec2.internal,},FirstTimestamp:2026-05-25 13:05:33.750327239 +0000 UTC m=+0.469290369,LastTimestamp:2026-05-25 13:05:33.750327239 +0000 UTC m=+0.469290369,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-151.ec2.internal,}"
May 25 13:05:33.766415 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.766400    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qnlb2"
May 25 13:05:33.789373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.789350    2559 manager.go:341] "Starting Device Plugin manager"
May 25 13:05:33.827505 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.789383    2559 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 25 13:05:33.827505 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.789396    2559 server.go:85] "Starting device plugin registration server"
May 25 13:05:33.827505 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.789659    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
May 25 13:05:33.827505 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.789671    2559 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 25 13:05:33.827505 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.789794    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 25 13:05:33.827505 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.789898    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 25 13:05:33.827505 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.789908    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 25 13:05:33.827505 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.790364    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 25 13:05:33.827505 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.790397    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-151.ec2.internal\" not found"
May 25 13:05:33.862912 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.862871    2559 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 25 13:05:33.864082 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.864066    2559 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 25 13:05:33.864163 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.864093    2559 status_manager.go:230] "Starting to sync pod status with apiserver"
May 25 13:05:33.864163 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.864119    2559 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 25 13:05:33.864163 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.864128    2559 kubelet.go:2451] "Starting kubelet main sync loop"
May 25 13:05:33.864309 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.864176    2559 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 25 13:05:33.867571 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.867506    2559 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 13:05:33.890288 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.890267    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 13:05:33.891177 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.891163    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasSufficientMemory"
May 25 13:05:33.891249 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.891191    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasNoDiskPressure"
May 25 13:05:33.891249 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.891202    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasSufficientPID"
May 25 13:05:33.891249 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.891225    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-151.ec2.internal"
May 25 13:05:33.898768 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.898753    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-151.ec2.internal"
May 25 13:05:33.898845 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.898774    2559 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-151.ec2.internal\": node \"ip-10-0-129-151.ec2.internal\" not found"
May 25 13:05:33.919988 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.919965    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-151.ec2.internal\" not found"
May 25 13:05:33.964824 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.964784    2559 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-151.ec2.internal"]
May 25 13:05:33.964978 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.964867    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 13:05:33.967495 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.967479    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasSufficientMemory"
May 25 13:05:33.967575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.967509    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasNoDiskPressure"
May 25 13:05:33.967575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.967520    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasSufficientPID"
May 25 13:05:33.968695 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.968684    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 13:05:33.968867 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.968851    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal"
May 25 13:05:33.968910 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.968884    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 13:05:33.969436 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.969421    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasSufficientMemory"
May 25 13:05:33.969436 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.969429    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasSufficientMemory"
May 25 13:05:33.969522 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.969447    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasNoDiskPressure"
May 25 13:05:33.969522 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.969456    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasSufficientPID"
May 25 13:05:33.969522 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.969465    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasNoDiskPressure"
May 25 13:05:33.969522 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.969476    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasSufficientPID"
May 25 13:05:33.970707 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.970691    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-151.ec2.internal"
May 25 13:05:33.970801 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.970730    2559 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 13:05:33.971411 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.971395    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasSufficientMemory"
May 25 13:05:33.971484 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.971423    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasNoDiskPressure"
May 25 13:05:33.971484 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:33.971432    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeHasSufficientPID"
May 25 13:05:33.992580 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.992551    2559 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-151.ec2.internal\" not found" node="ip-10-0-129-151.ec2.internal"
May 25 13:05:33.997185 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:33.997164    2559 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-151.ec2.internal\" not found" node="ip-10-0-129-151.ec2.internal"
May 25 13:05:34.021012 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:34.020986    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-151.ec2.internal\" not found"
May 25 13:05:34.032866 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.032847    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/709c800460854ebcab9b23cb49bcdc37-config\") pod \"kube-apiserver-proxy-ip-10-0-129-151.ec2.internal\" (UID: \"709c800460854ebcab9b23cb49bcdc37\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-151.ec2.internal"
May 25 13:05:34.032935 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.032894    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8bc72641362eb99e2360a0badc226d4a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal\" (UID: \"8bc72641362eb99e2360a0badc226d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal"
May 25 13:05:34.032935 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.032923    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8bc72641362eb99e2360a0badc226d4a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal\" (UID: \"8bc72641362eb99e2360a0badc226d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal"
May 25 13:05:34.121638 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:34.121563    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-151.ec2.internal\" not found"
May 25 13:05:34.133921 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.133898    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8bc72641362eb99e2360a0badc226d4a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal\" (UID: \"8bc72641362eb99e2360a0badc226d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal"
May 25 13:05:34.134000 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.133925    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/709c800460854ebcab9b23cb49bcdc37-config\") pod \"kube-apiserver-proxy-ip-10-0-129-151.ec2.internal\" (UID: \"709c800460854ebcab9b23cb49bcdc37\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-151.ec2.internal"
May 25 13:05:34.134000 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.133941    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8bc72641362eb99e2360a0badc226d4a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal\" (UID: \"8bc72641362eb99e2360a0badc226d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal"
May 25 13:05:34.134000 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.133978    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8bc72641362eb99e2360a0badc226d4a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal\" (UID: \"8bc72641362eb99e2360a0badc226d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal"
May 25 13:05:34.134113 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.134001    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/709c800460854ebcab9b23cb49bcdc37-config\") pod \"kube-apiserver-proxy-ip-10-0-129-151.ec2.internal\" (UID: \"709c800460854ebcab9b23cb49bcdc37\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-151.ec2.internal"
May 25 13:05:34.134113 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.134009    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8bc72641362eb99e2360a0badc226d4a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal\" (UID: \"8bc72641362eb99e2360a0badc226d4a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal"
May 25 13:05:34.222353 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:34.222319    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-151.ec2.internal\" not found"
May 25 13:05:34.295910 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.295860    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal"
May 25 13:05:34.299687 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.299663    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-151.ec2.internal"
May 25 13:05:34.323293 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:34.323268    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-151.ec2.internal\" not found"
May 25 13:05:34.423937 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:34.423851    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-151.ec2.internal\" not found"
May 25 13:05:34.524346 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:34.524312    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-151.ec2.internal\" not found"
May 25 13:05:34.625003 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:34.624968    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-151.ec2.internal\" not found"
May 25 13:05:34.637271 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.637250    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 25 13:05:34.637394 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.637380    2559 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 25 13:05:34.725587 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:34.725525    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-151.ec2.internal\" not found"
May 25 13:05:34.731510 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.731490    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 25 13:05:34.743531 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.743511    2559 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 25 13:05:34.757802 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:34.757752    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc72641362eb99e2360a0badc226d4a.slice/crio-9bb5d5627df7a863f3dee45fd31da579ac4d7b9644cca451a083cfa0e3dadf8e WatchSource:0}: Error finding container 9bb5d5627df7a863f3dee45fd31da579ac4d7b9644cca451a083cfa0e3dadf8e: Status 404 returned error can't find the container with id 9bb5d5627df7a863f3dee45fd31da579ac4d7b9644cca451a083cfa0e3dadf8e
May 25 13:05:34.760773 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.760757    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 13:05:34.764943 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.764927    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f5l46"
May 25 13:05:34.768000 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.767973    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-24 13:00:33 +0000 UTC" deadline="2028-01-14 08:50:27.629651448 +0000 UTC"
May 25 13:05:34.768000 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.768000    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14371h44m52.861654712s"
May 25 13:05:34.772868 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.772847    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f5l46"
May 25 13:05:34.826168 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:34.826130    2559 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-129-151.ec2.internal\" not found"
May 25 13:05:34.844021 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.843993    2559 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 13:05:34.867885 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.867844    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal" event={"ID":"8bc72641362eb99e2360a0badc226d4a","Type":"ContainerStarted","Data":"9bb5d5627df7a863f3dee45fd31da579ac4d7b9644cca451a083cfa0e3dadf8e"}
May 25 13:05:34.898438 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.898419    2559 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 13:05:34.931565 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.931536    2559 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-151.ec2.internal"
May 25 13:05:34.941975 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.941958    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 25 13:05:34.943800 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.943785    2559 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal"
May 25 13:05:34.947626 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.946744    2559 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 13:05:34.953363 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:34.953347    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 25 13:05:34.978165 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:34.978141    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod709c800460854ebcab9b23cb49bcdc37.slice/crio-c52067a9bfef802c45199be6d6acca3cd954ae42c64cc4ea8c7e8e6d2c9429ad WatchSource:0}: Error finding container c52067a9bfef802c45199be6d6acca3cd954ae42c64cc4ea8c7e8e6d2c9429ad: Status 404 returned error can't find the container with id c52067a9bfef802c45199be6d6acca3cd954ae42c64cc4ea8c7e8e6d2c9429ad
May 25 13:05:35.708470 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.708436    2559 apiserver.go:52] "Watching apiserver"
May 25 13:05:35.715131 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.715104    2559 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 25 13:05:35.715518 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.715491    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-151.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal","openshift-multus/multus-tghhw","openshift-network-diagnostics/network-check-target-z7xmf","openshift-network-operator/iptables-alerter-v9xkm","kube-system/konnectivity-agent-zjh8s","openshift-cluster-node-tuning-operator/tuned-m6244","openshift-image-registry/node-ca-95pwd","openshift-multus/multus-additional-cni-plugins-9lvfw","openshift-multus/network-metrics-daemon-tx5hr","openshift-ovn-kubernetes/ovnkube-node-x4vqs"]
May 25 13:05:35.717134 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.717114    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.718362 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.718340    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tghhw"
May 25 13:05:35.719528 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.719506    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:35.719623 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:35.719581    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:05:35.720329 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.720041    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 25 13:05:35.720329 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.720082    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-k242x\""
May 25 13:05:35.720329 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.720221    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 25 13:05:35.720329 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.720271    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 25 13:05:35.720614 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.720597    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 25 13:05:35.720714 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.720692    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 25 13:05:35.720967 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.720854    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 25 13:05:35.721366 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.720938    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ljhgm\""
May 25 13:05:35.721562 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.721548    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 25 13:05:35.722421 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.722386    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v9xkm"
May 25 13:05:35.724034 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.723793    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zjh8s"
May 25 13:05:35.724452 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.724415    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 25 13:05:35.724535 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.724493    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kfwjb\""
May 25 13:05:35.724955 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.724917    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 25 13:05:35.725408 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.725203    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 25 13:05:35.725408 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.725400    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:35.725562 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:35.725458    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:05:35.726142 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.725988    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 25 13:05:35.726883 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.726692    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 25 13:05:35.726883 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.726703    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pc95k\""
May 25 13:05:35.726883 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.726807    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.727357 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.727050    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.728952 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.728288    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.730453 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.729888    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 25 13:05:35.730453 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.729951    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 25 13:05:35.730453 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.729894    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 25 13:05:35.730453 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.730115    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cdfqp\""
May 25 13:05:35.730453 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.730154    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bsq9r\""
May 25 13:05:35.730453 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.730267    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 25 13:05:35.730453 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.730324    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 25 13:05:35.730453 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.730333    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 25 13:05:35.730453 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.730340    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 25 13:05:35.734206 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.731303    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 25 13:05:35.734206 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.731521    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 25 13:05:35.734206 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.731824    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tklm8\""
May 25 13:05:35.734206 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.731837    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 25 13:05:35.734577 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.734561    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-95pwd"
May 25 13:05:35.736734 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.736585    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 25 13:05:35.736734 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.736602    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 25 13:05:35.736734 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.736697    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9x8rn\""
May 25 13:05:35.737201 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.737012    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 25 13:05:35.741302 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741282    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-systemd-units\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.741402 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741316    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-run-systemd\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.741402 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741342    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-run-ovn-kubernetes\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.741402 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741366    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-run\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.741402 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741397    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-log-socket\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.741600 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741421    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkp7v\" (UniqueName: \"kubernetes.io/projected/bc716b46-7c31-42f5-ba6f-45b8c77d29ad-kube-api-access-lkp7v\") pod \"node-ca-95pwd\" (UID: \"bc716b46-7c31-42f5-ba6f-45b8c77d29ad\") " pod="openshift-image-registry/node-ca-95pwd"
May 25 13:05:35.741600 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741446    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-registration-dir\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.741600 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741468    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-etc-selinux\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.741600 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741495    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/03bb8722-1d72-4446-8a56-7d37772243a4-konnectivity-ca\") pod \"konnectivity-agent-zjh8s\" (UID: \"03bb8722-1d72-4446-8a56-7d37772243a4\") " pod="kube-system/konnectivity-agent-zjh8s"
May 25 13:05:35.741600 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741518    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bwpf\" (UniqueName: \"kubernetes.io/projected/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-kube-api-access-2bwpf\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.741600 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741540    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/913e5f01-6ff3-4bd5-965e-809823df7d6d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.741600 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741563    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/913e5f01-6ff3-4bd5-965e-809823df7d6d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.741600 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741585    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-hostroot\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741609    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1a62b5a7-d7eb-4389-a105-e8b1e06b1604-iptables-alerter-script\") pod \"iptables-alerter-v9xkm\" (UID: \"1a62b5a7-d7eb-4389-a105-e8b1e06b1604\") " pod="openshift-network-operator/iptables-alerter-v9xkm"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741630    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-lib-modules\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741651    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/54e5ea0f-6319-42c2-9770-76a93d975631-etc-tuned\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741677    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc492\" (UniqueName: \"kubernetes.io/projected/c886a9b9-c5da-4569-a695-125c80d706f6-kube-api-access-sc492\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741702    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/03bb8722-1d72-4446-8a56-7d37772243a4-agent-certs\") pod \"konnectivity-agent-zjh8s\" (UID: \"03bb8722-1d72-4446-8a56-7d37772243a4\") " pod="kube-system/konnectivity-agent-zjh8s"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-ovn-node-metrics-cert\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741763    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-socket-dir\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741793    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/913e5f01-6ff3-4bd5-965e-809823df7d6d-os-release\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741818    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-var-lib-kubelet\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741839    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-multus-cni-dir\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741864    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqhqp\" (UniqueName: \"kubernetes.io/projected/b2ceae0e-518e-4e3c-9892-d50a78b74642-kube-api-access-cqhqp\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741894    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwrm2\" (UniqueName: \"kubernetes.io/projected/fde8fc4f-07bb-442f-aa5e-6e63efece856-kube-api-access-cwrm2\") pod \"network-metrics-daemon-tx5hr\" (UID: \"fde8fc4f-07bb-442f-aa5e-6e63efece856\") " pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741917    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-slash\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741939    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc716b46-7c31-42f5-ba6f-45b8c77d29ad-host\") pod \"node-ca-95pwd\" (UID: \"bc716b46-7c31-42f5-ba6f-45b8c77d29ad\") " pod="openshift-image-registry/node-ca-95pwd"
May 25 13:05:35.741991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.741983    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bc716b46-7c31-42f5-ba6f-45b8c77d29ad-serviceca\") pod \"node-ca-95pwd\" (UID: \"bc716b46-7c31-42f5-ba6f-45b8c77d29ad\") " pod="openshift-image-registry/node-ca-95pwd"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742020    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/913e5f01-6ff3-4bd5-965e-809823df7d6d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742057    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-multus-socket-dir-parent\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742085    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-var-lib-cni-bin\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742115    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-run-netns\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742184    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/913e5f01-6ff3-4bd5-965e-809823df7d6d-cnibin\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742215    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fccv4\" (UniqueName: \"kubernetes.io/projected/913e5f01-6ff3-4bd5-965e-809823df7d6d-kube-api-access-fccv4\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742260    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-modprobe-d\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742289    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-systemd\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742349    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-sys\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742377    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-var-lib-kubelet\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742407    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-env-overrides\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742430    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-ovnkube-script-lib\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742454    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/913e5f01-6ff3-4bd5-965e-809823df7d6d-system-cni-dir\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742496    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-var-lib-cni-multus\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742531    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-multus-conf-dir\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.742628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742554    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-etc-kubernetes\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742577    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-cni-bin\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742602    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-kubelet-dir\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742641    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9bw4\" (UniqueName: \"kubernetes.io/projected/1a62b5a7-d7eb-4389-a105-e8b1e06b1604-kube-api-access-h9bw4\") pod \"iptables-alerter-v9xkm\" (UID: \"1a62b5a7-d7eb-4389-a105-e8b1e06b1604\") " pod="openshift-network-operator/iptables-alerter-v9xkm"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742664    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-sysctl-d\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742686    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-system-cni-dir\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742708    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-cnibin\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742748    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-run-k8s-cni-cncf-io\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742770    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-ovnkube-config\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742805    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-sysconfig\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742834    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-sysctl-conf\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742858    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-host\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742880    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c886a9b9-c5da-4569-a695-125c80d706f6-cni-binary-copy\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742902    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c886a9b9-c5da-4569-a695-125c80d706f6-multus-daemon-config\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742924    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-kubelet\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742946    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-var-lib-openvswitch\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742969    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-run-ovn\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.743373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.742990    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-sys-fs\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743014    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k99qx\" (UniqueName: \"kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx\") pod \"network-check-target-z7xmf\" (UID: \"d4c3716c-3120-42b7-a927-bca86e4b5383\") " pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743036    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-kubernetes\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743060    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-cni-netd\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743084    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/913e5f01-6ff3-4bd5-965e-809823df7d6d-cni-binary-copy\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743106    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2wg2\" (UniqueName: \"kubernetes.io/projected/54e5ea0f-6319-42c2-9770-76a93d975631-kube-api-access-h2wg2\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743128    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-run-netns\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743164    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-run-openvswitch\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743189    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-node-log\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743215    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743238    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-device-dir\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743259    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a62b5a7-d7eb-4389-a105-e8b1e06b1604-host-slash\") pod \"iptables-alerter-v9xkm\" (UID: \"1a62b5a7-d7eb-4389-a105-e8b1e06b1604\") " pod="openshift-network-operator/iptables-alerter-v9xkm"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743280    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/54e5ea0f-6319-42c2-9770-76a93d975631-tmp\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743304    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-os-release\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743325    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-etc-openvswitch\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743347    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs\") pod \"network-metrics-daemon-tx5hr\" (UID: \"fde8fc4f-07bb-442f-aa5e-6e63efece856\") " pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:35.744091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.743375    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-run-multus-certs\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.773597 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.773564    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-24 13:00:34 +0000 UTC" deadline="2027-12-25 21:12:24.226061038 +0000 UTC"
May 25 13:05:35.773707 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.773599    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13904h6m48.452466024s"
May 25 13:05:35.833060 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.833031    2559 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 25 13:05:35.843997 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.843964    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/913e5f01-6ff3-4bd5-965e-809823df7d6d-cnibin\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.843997 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844000    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fccv4\" (UniqueName: \"kubernetes.io/projected/913e5f01-6ff3-4bd5-965e-809823df7d6d-kube-api-access-fccv4\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844021    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-modprobe-d\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844045    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-systemd\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844066    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-sys\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844075    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/913e5f01-6ff3-4bd5-965e-809823df7d6d-cnibin\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844087    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-var-lib-kubelet\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844110    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-env-overrides\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844116    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-systemd\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844140    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-sys\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844146    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-ovnkube-script-lib\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844161    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-modprobe-d\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844171    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/913e5f01-6ff3-4bd5-965e-809823df7d6d-system-cni-dir\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844176    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-var-lib-kubelet\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844195    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-var-lib-cni-multus\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.844221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844227    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-var-lib-cni-multus\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844232    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/913e5f01-6ff3-4bd5-965e-809823df7d6d-system-cni-dir\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844252    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-multus-conf-dir\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844272    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-etc-kubernetes\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844288    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-cni-bin\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844326    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-cni-bin\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844340    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-multus-conf-dir\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844385    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-kubelet-dir\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844417    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h9bw4\" (UniqueName: \"kubernetes.io/projected/1a62b5a7-d7eb-4389-a105-e8b1e06b1604-kube-api-access-h9bw4\") pod \"iptables-alerter-v9xkm\" (UID: \"1a62b5a7-d7eb-4389-a105-e8b1e06b1604\") " pod="openshift-network-operator/iptables-alerter-v9xkm"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844442    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-sysctl-d\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844467    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-system-cni-dir\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844492    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-cnibin\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844517    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-run-k8s-cni-cncf-io\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844541    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-ovnkube-config\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844566    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-sysconfig\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844602    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-env-overrides\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844614    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-sysctl-d\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844590    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-sysctl-conf\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.844870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844673    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-run-k8s-cni-cncf-io\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844699    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-host\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844745    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c886a9b9-c5da-4569-a695-125c80d706f6-cni-binary-copy\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844749    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-sysctl-conf\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844757    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-ovnkube-script-lib\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844750    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-system-cni-dir\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844799    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-host\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844807    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c886a9b9-c5da-4569-a695-125c80d706f6-multus-daemon-config\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844337    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-etc-kubernetes\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844819    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-cnibin\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844853    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-kubelet\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844855    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-sysconfig\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844886    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-var-lib-openvswitch\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844913    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-run-ovn\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844937    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-sys-fs\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844946    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-var-lib-openvswitch\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844912    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-kubelet\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844960    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k99qx\" (UniqueName: \"kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx\") pod \"network-check-target-z7xmf\" (UID: \"d4c3716c-3120-42b7-a927-bca86e4b5383\") " pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:35.845519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844964    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-kubelet-dir\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844983    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-run-ovn\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.844992    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-kubernetes\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845032    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-cni-netd\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845033    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-sys-fs\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845056    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/913e5f01-6ff3-4bd5-965e-809823df7d6d-cni-binary-copy\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845082    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h2wg2\" (UniqueName: \"kubernetes.io/projected/54e5ea0f-6319-42c2-9770-76a93d975631-kube-api-access-h2wg2\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845108    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-run-netns\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845143    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-run-openvswitch\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845082    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-etc-kubernetes\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845112    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-cni-netd\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845194    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-node-log\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845231    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845259    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-device-dir\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845275    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-ovnkube-config\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845283    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a62b5a7-d7eb-4389-a105-e8b1e06b1604-host-slash\") pod \"iptables-alerter-v9xkm\" (UID: \"1a62b5a7-d7eb-4389-a105-e8b1e06b1604\") " pod="openshift-network-operator/iptables-alerter-v9xkm"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845332    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1a62b5a7-d7eb-4389-a105-e8b1e06b1604-host-slash\") pod \"iptables-alerter-v9xkm\" (UID: \"1a62b5a7-d7eb-4389-a105-e8b1e06b1604\") " pod="openshift-network-operator/iptables-alerter-v9xkm"
May 25 13:05:35.846258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845336    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-device-dir\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845350    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/54e5ea0f-6319-42c2-9770-76a93d975631-tmp\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845372    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-node-log\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845375    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-run-openvswitch\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845397    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c886a9b9-c5da-4569-a695-125c80d706f6-cni-binary-copy\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845474    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c886a9b9-c5da-4569-a695-125c80d706f6-multus-daemon-config\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845622    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845650    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-os-release\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845665    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/913e5f01-6ff3-4bd5-965e-809823df7d6d-cni-binary-copy\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845678    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-etc-openvswitch\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845705    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs\") pod \"network-metrics-daemon-tx5hr\" (UID: \"fde8fc4f-07bb-442f-aa5e-6e63efece856\") " pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845707    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-run-netns\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845744    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-run-multus-certs\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845750    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-os-release\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845794    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-systemd-units\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845798    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-run-multus-certs\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845813    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-etc-openvswitch\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845821    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-run-systemd\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.846977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845852    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-run-ovn-kubernetes\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845867    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-systemd-units\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:35.845883    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845901    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-run\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.845871    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-run\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:35.846461    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs podName:fde8fc4f-07bb-442f-aa5e-6e63efece856 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:36.346409425 +0000 UTC m=+3.065372565 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs") pod "network-metrics-daemon-tx5hr" (UID: "fde8fc4f-07bb-442f-aa5e-6e63efece856") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.846592    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-run-systemd\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.846639    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-run-ovn-kubernetes\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.846682    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-log-socket\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.846750    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lkp7v\" (UniqueName: \"kubernetes.io/projected/bc716b46-7c31-42f5-ba6f-45b8c77d29ad-kube-api-access-lkp7v\") pod \"node-ca-95pwd\" (UID: \"bc716b46-7c31-42f5-ba6f-45b8c77d29ad\") " pod="openshift-image-registry/node-ca-95pwd"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.846783    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-registration-dir\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.846931    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-etc-selinux\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.846961    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/03bb8722-1d72-4446-8a56-7d37772243a4-konnectivity-ca\") pod \"konnectivity-agent-zjh8s\" (UID: \"03bb8722-1d72-4446-8a56-7d37772243a4\") " pod="kube-system/konnectivity-agent-zjh8s"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.846991    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2bwpf\" (UniqueName: \"kubernetes.io/projected/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-kube-api-access-2bwpf\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.846980    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.847284    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/913e5f01-6ff3-4bd5-965e-809823df7d6d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.847327    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/913e5f01-6ff3-4bd5-965e-809823df7d6d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.847748 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.847357    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-hostroot\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.847364    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-etc-selinux\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.847392    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1a62b5a7-d7eb-4389-a105-e8b1e06b1604-iptables-alerter-script\") pod \"iptables-alerter-v9xkm\" (UID: \"1a62b5a7-d7eb-4389-a105-e8b1e06b1604\") " pod="openshift-network-operator/iptables-alerter-v9xkm"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.847426    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-log-socket\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.847575    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-hostroot\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.847586    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/913e5f01-6ff3-4bd5-965e-809823df7d6d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.847636    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-lib-modules\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.847637    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-registration-dir\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848095    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/913e5f01-6ff3-4bd5-965e-809823df7d6d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848163    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1a62b5a7-d7eb-4389-a105-e8b1e06b1604-iptables-alerter-script\") pod \"iptables-alerter-v9xkm\" (UID: \"1a62b5a7-d7eb-4389-a105-e8b1e06b1604\") " pod="openshift-network-operator/iptables-alerter-v9xkm"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848217    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-lib-modules\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848219    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/54e5ea0f-6319-42c2-9770-76a93d975631-etc-tuned\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848259    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sc492\" (UniqueName: \"kubernetes.io/projected/c886a9b9-c5da-4569-a695-125c80d706f6-kube-api-access-sc492\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848288    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/03bb8722-1d72-4446-8a56-7d37772243a4-agent-certs\") pod \"konnectivity-agent-zjh8s\" (UID: \"03bb8722-1d72-4446-8a56-7d37772243a4\") " pod="kube-system/konnectivity-agent-zjh8s"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848318    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-ovn-node-metrics-cert\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848348    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-socket-dir\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848371    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/913e5f01-6ff3-4bd5-965e-809823df7d6d-os-release\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.848447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848399    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-var-lib-kubelet\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.849152 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848426    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-multus-cni-dir\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.849152 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848454    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cqhqp\" (UniqueName: \"kubernetes.io/projected/b2ceae0e-518e-4e3c-9892-d50a78b74642-kube-api-access-cqhqp\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.849152 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848483    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cwrm2\" (UniqueName: \"kubernetes.io/projected/fde8fc4f-07bb-442f-aa5e-6e63efece856-kube-api-access-cwrm2\") pod \"network-metrics-daemon-tx5hr\" (UID: \"fde8fc4f-07bb-442f-aa5e-6e63efece856\") " pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:35.849152 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848506    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-slash\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.849152 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848533    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc716b46-7c31-42f5-ba6f-45b8c77d29ad-host\") pod \"node-ca-95pwd\" (UID: \"bc716b46-7c31-42f5-ba6f-45b8c77d29ad\") " pod="openshift-image-registry/node-ca-95pwd"
May 25 13:05:35.849152 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848558    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bc716b46-7c31-42f5-ba6f-45b8c77d29ad-serviceca\") pod \"node-ca-95pwd\" (UID: \"bc716b46-7c31-42f5-ba6f-45b8c77d29ad\") " pod="openshift-image-registry/node-ca-95pwd"
May 25 13:05:35.849152 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848587    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/913e5f01-6ff3-4bd5-965e-809823df7d6d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.849152 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848618    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-multus-socket-dir-parent\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.849152 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848642    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-var-lib-cni-bin\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.849152 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848669    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-run-netns\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.849152 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.848764    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-run-netns\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.850904 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.850118    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/03bb8722-1d72-4446-8a56-7d37772243a4-konnectivity-ca\") pod \"konnectivity-agent-zjh8s\" (UID: \"03bb8722-1d72-4446-8a56-7d37772243a4\") " pod="kube-system/konnectivity-agent-zjh8s"
May 25 13:05:35.850904 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.850500    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bc716b46-7c31-42f5-ba6f-45b8c77d29ad-serviceca\") pod \"node-ca-95pwd\" (UID: \"bc716b46-7c31-42f5-ba6f-45b8c77d29ad\") " pod="openshift-image-registry/node-ca-95pwd"
May 25 13:05:35.850904 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.850579    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-host-slash\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.850904 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.850582    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-host-var-lib-cni-bin\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.850904 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.850592    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bc716b46-7c31-42f5-ba6f-45b8c77d29ad-host\") pod \"node-ca-95pwd\" (UID: \"bc716b46-7c31-42f5-ba6f-45b8c77d29ad\") " pod="openshift-image-registry/node-ca-95pwd"
May 25 13:05:35.850904 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.850579    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54e5ea0f-6319-42c2-9770-76a93d975631-var-lib-kubelet\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.850904 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.850665    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-multus-cni-dir\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.850904 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.850669    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2ceae0e-518e-4e3c-9892-d50a78b74642-socket-dir\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.850904 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.850740    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c886a9b9-c5da-4569-a695-125c80d706f6-multus-socket-dir-parent\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.850904 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.850763    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/913e5f01-6ff3-4bd5-965e-809823df7d6d-os-release\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.851403 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.851212    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/913e5f01-6ff3-4bd5-965e-809823df7d6d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.851403 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.851272    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/54e5ea0f-6319-42c2-9770-76a93d975631-etc-tuned\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.853005 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.852961    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/54e5ea0f-6319-42c2-9770-76a93d975631-tmp\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.854735 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.854316    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/03bb8722-1d72-4446-8a56-7d37772243a4-agent-certs\") pod \"konnectivity-agent-zjh8s\" (UID: \"03bb8722-1d72-4446-8a56-7d37772243a4\") " pod="kube-system/konnectivity-agent-zjh8s"
May 25 13:05:35.855483 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:35.855348    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 13:05:35.855483 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:35.855369    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 13:05:35.855483 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:35.855382    2559 projected.go:194] Error preparing data for projected volume kube-api-access-k99qx for pod openshift-network-diagnostics/network-check-target-z7xmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:05:35.855483 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:35.855439    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx podName:d4c3716c-3120-42b7-a927-bca86e4b5383 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:36.355421007 +0000 UTC m=+3.074384145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k99qx" (UniqueName: "kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx") pod "network-check-target-z7xmf" (UID: "d4c3716c-3120-42b7-a927-bca86e4b5383") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:05:35.855757 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.855537    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-ovn-node-metrics-cert\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.856386 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.856361    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fccv4\" (UniqueName: \"kubernetes.io/projected/913e5f01-6ff3-4bd5-965e-809823df7d6d-kube-api-access-fccv4\") pod \"multus-additional-cni-plugins-9lvfw\" (UID: \"913e5f01-6ff3-4bd5-965e-809823df7d6d\") " pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:35.856918 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.856894    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9bw4\" (UniqueName: \"kubernetes.io/projected/1a62b5a7-d7eb-4389-a105-e8b1e06b1604-kube-api-access-h9bw4\") pod \"iptables-alerter-v9xkm\" (UID: \"1a62b5a7-d7eb-4389-a105-e8b1e06b1604\") " pod="openshift-network-operator/iptables-alerter-v9xkm"
May 25 13:05:35.857693 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.857670    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkp7v\" (UniqueName: \"kubernetes.io/projected/bc716b46-7c31-42f5-ba6f-45b8c77d29ad-kube-api-access-lkp7v\") pod \"node-ca-95pwd\" (UID: \"bc716b46-7c31-42f5-ba6f-45b8c77d29ad\") " pod="openshift-image-registry/node-ca-95pwd"
May 25 13:05:35.858342 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.858299    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc492\" (UniqueName: \"kubernetes.io/projected/c886a9b9-c5da-4569-a695-125c80d706f6-kube-api-access-sc492\") pod \"multus-tghhw\" (UID: \"c886a9b9-c5da-4569-a695-125c80d706f6\") " pod="openshift-multus/multus-tghhw"
May 25 13:05:35.859026 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.858955    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bwpf\" (UniqueName: \"kubernetes.io/projected/724bf4c7-1d41-4ddf-aa5e-d8b043c08c35-kube-api-access-2bwpf\") pod \"ovnkube-node-x4vqs\" (UID: \"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35\") " pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:35.859482 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.859438    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwrm2\" (UniqueName: \"kubernetes.io/projected/fde8fc4f-07bb-442f-aa5e-6e63efece856-kube-api-access-cwrm2\") pod \"network-metrics-daemon-tx5hr\" (UID: \"fde8fc4f-07bb-442f-aa5e-6e63efece856\") " pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:35.859676 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.859658    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqhqp\" (UniqueName: \"kubernetes.io/projected/b2ceae0e-518e-4e3c-9892-d50a78b74642-kube-api-access-cqhqp\") pod \"aws-ebs-csi-driver-node-76tgv\" (UID: \"b2ceae0e-518e-4e3c-9892-d50a78b74642\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:35.860563 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.860544    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2wg2\" (UniqueName: \"kubernetes.io/projected/54e5ea0f-6319-42c2-9770-76a93d975631-kube-api-access-h2wg2\") pod \"tuned-m6244\" (UID: \"54e5ea0f-6319-42c2-9770-76a93d975631\") " pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:35.870307 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.870274    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-151.ec2.internal" event={"ID":"709c800460854ebcab9b23cb49bcdc37","Type":"ContainerStarted","Data":"c52067a9bfef802c45199be6d6acca3cd954ae42c64cc4ea8c7e8e6d2c9429ad"}
May 25 13:05:35.878004 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:35.877984    2559 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 13:05:36.033368 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.033291    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv"
May 25 13:05:36.042984 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.042951    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tghhw"
May 25 13:05:36.053653 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.053629    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v9xkm"
May 25 13:05:36.060236 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.060216    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zjh8s"
May 25 13:05:36.068816 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.068795    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9lvfw"
May 25 13:05:36.077447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.077426    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:05:36.085065 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.085042    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m6244"
May 25 13:05:36.091605 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.091580    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-95pwd"
May 25 13:05:36.351420 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.351344    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs\") pod \"network-metrics-daemon-tx5hr\" (UID: \"fde8fc4f-07bb-442f-aa5e-6e63efece856\") " pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:36.351570 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:36.351472    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:05:36.351570 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:36.351531    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs podName:fde8fc4f-07bb-442f-aa5e-6e63efece856 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:37.351516378 +0000 UTC m=+4.070479495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs") pod "network-metrics-daemon-tx5hr" (UID: "fde8fc4f-07bb-442f-aa5e-6e63efece856") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:05:36.404075 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:36.403599    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod724bf4c7_1d41_4ddf_aa5e_d8b043c08c35.slice/crio-e29837d1d756e52978817f6a613b922dd4e64484942b38a0982740af467438dd WatchSource:0}: Error finding container e29837d1d756e52978817f6a613b922dd4e64484942b38a0982740af467438dd: Status 404 returned error can't find the container with id e29837d1d756e52978817f6a613b922dd4e64484942b38a0982740af467438dd
May 25 13:05:36.404842 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:36.404816    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc716b46_7c31_42f5_ba6f_45b8c77d29ad.slice/crio-4bfe5481febd237c2311117bf39059a1fcac0ff403de87819440d70a428d58f0 WatchSource:0}: Error finding container 4bfe5481febd237c2311117bf39059a1fcac0ff403de87819440d70a428d58f0: Status 404 returned error can't find the container with id 4bfe5481febd237c2311117bf39059a1fcac0ff403de87819440d70a428d58f0
May 25 13:05:36.405601 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:36.405572    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a62b5a7_d7eb_4389_a105_e8b1e06b1604.slice/crio-0028ea111a58e6896550770d97aa8a19134d59951c16876a0c83fa9fb9857ffb WatchSource:0}: Error finding container 0028ea111a58e6896550770d97aa8a19134d59951c16876a0c83fa9fb9857ffb: Status 404 returned error can't find the container with id 0028ea111a58e6896550770d97aa8a19134d59951c16876a0c83fa9fb9857ffb
May 25 13:05:36.406600 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:36.406475    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod913e5f01_6ff3_4bd5_965e_809823df7d6d.slice/crio-5bdbebe247ff679b61d7747d38875c52f236f010bf00a0cac0fba9999745296e WatchSource:0}: Error finding container 5bdbebe247ff679b61d7747d38875c52f236f010bf00a0cac0fba9999745296e: Status 404 returned error can't find the container with id 5bdbebe247ff679b61d7747d38875c52f236f010bf00a0cac0fba9999745296e
May 25 13:05:36.410842 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:36.410817    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54e5ea0f_6319_42c2_9770_76a93d975631.slice/crio-c31c0ff610bb0f858b46f4cc4fbe1f3371f972e75efd65958d712005aa77eeae WatchSource:0}: Error finding container c31c0ff610bb0f858b46f4cc4fbe1f3371f972e75efd65958d712005aa77eeae: Status 404 returned error can't find the container with id c31c0ff610bb0f858b46f4cc4fbe1f3371f972e75efd65958d712005aa77eeae
May 25 13:05:36.412128 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:36.411860    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2ceae0e_518e_4e3c_9892_d50a78b74642.slice/crio-a1d99219b7f9aed7efd6e1e95fc4a7452464b8fd3628ec213455fb5f9d65d6ca WatchSource:0}: Error finding container a1d99219b7f9aed7efd6e1e95fc4a7452464b8fd3628ec213455fb5f9d65d6ca: Status 404 returned error can't find the container with id a1d99219b7f9aed7efd6e1e95fc4a7452464b8fd3628ec213455fb5f9d65d6ca
May 25 13:05:36.413225 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:36.413150    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc886a9b9_c5da_4569_a695_125c80d706f6.slice/crio-4758e3a2a2fb2e9e420367937dcc34acc23e7508ad01cabac39d8264b2a880b3 WatchSource:0}: Error finding container 4758e3a2a2fb2e9e420367937dcc34acc23e7508ad01cabac39d8264b2a880b3: Status 404 returned error can't find the container with id 4758e3a2a2fb2e9e420367937dcc34acc23e7508ad01cabac39d8264b2a880b3
May 25 13:05:36.413451 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:05:36.413429    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03bb8722_1d72_4446_8a56_7d37772243a4.slice/crio-19c6f061224e019c64f2408d99d5bdeb717629ccf9b38e3c7ea005ae500ab33d WatchSource:0}: Error finding container 19c6f061224e019c64f2408d99d5bdeb717629ccf9b38e3c7ea005ae500ab33d: Status 404 returned error can't find the container with id 19c6f061224e019c64f2408d99d5bdeb717629ccf9b38e3c7ea005ae500ab33d
May 25 13:05:36.451606 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.451580    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k99qx\" (UniqueName: \"kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx\") pod \"network-check-target-z7xmf\" (UID: \"d4c3716c-3120-42b7-a927-bca86e4b5383\") " pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:36.451747 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:36.451730    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 13:05:36.451803 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:36.451752    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 13:05:36.451803 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:36.451763    2559 projected.go:194] Error preparing data for projected volume kube-api-access-k99qx for pod openshift-network-diagnostics/network-check-target-z7xmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:05:36.451876 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:36.451819    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx podName:d4c3716c-3120-42b7-a927-bca86e4b5383 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:37.451805422 +0000 UTC m=+4.170768542 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k99qx" (UniqueName: "kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx") pod "network-check-target-z7xmf" (UID: "d4c3716c-3120-42b7-a927-bca86e4b5383") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:05:36.774433 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.774212    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-24 13:00:34 +0000 UTC" deadline="2027-11-22 14:05:12.373281226 +0000 UTC"
May 25 13:05:36.774433 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.774406    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13104h59m35.598880311s"
May 25 13:05:36.877633 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.877571    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-95pwd" event={"ID":"bc716b46-7c31-42f5-ba6f-45b8c77d29ad","Type":"ContainerStarted","Data":"4bfe5481febd237c2311117bf39059a1fcac0ff403de87819440d70a428d58f0"}
May 25 13:05:36.879945 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.879879    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs" event={"ID":"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35","Type":"ContainerStarted","Data":"e29837d1d756e52978817f6a613b922dd4e64484942b38a0982740af467438dd"}
May 25 13:05:36.884645 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.883830    2559 generic.go:358] "Generic (PLEG): container finished" podID="8bc72641362eb99e2360a0badc226d4a" containerID="61af3a5140bc78cb574d3837c160a3bc5dc0435cf86676a351dfc705d6fe78df" exitCode=0
May 25 13:05:36.884645 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.883901    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal" event={"ID":"8bc72641362eb99e2360a0badc226d4a","Type":"ContainerDied","Data":"61af3a5140bc78cb574d3837c160a3bc5dc0435cf86676a351dfc705d6fe78df"}
May 25 13:05:36.887819 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.887796    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tghhw" event={"ID":"c886a9b9-c5da-4569-a695-125c80d706f6","Type":"ContainerStarted","Data":"4758e3a2a2fb2e9e420367937dcc34acc23e7508ad01cabac39d8264b2a880b3"}
May 25 13:05:36.891073 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.891029    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m6244" event={"ID":"54e5ea0f-6319-42c2-9770-76a93d975631","Type":"ContainerStarted","Data":"c31c0ff610bb0f858b46f4cc4fbe1f3371f972e75efd65958d712005aa77eeae"}
May 25 13:05:36.893445 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.893358    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-151.ec2.internal" event={"ID":"709c800460854ebcab9b23cb49bcdc37","Type":"ContainerStarted","Data":"4ca36ef668e1680a0b45780c30234cf3dcf5f5b61c2b156944c2312924854051"}
May 25 13:05:36.895606 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.895568    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv" event={"ID":"b2ceae0e-518e-4e3c-9892-d50a78b74642","Type":"ContainerStarted","Data":"a1d99219b7f9aed7efd6e1e95fc4a7452464b8fd3628ec213455fb5f9d65d6ca"}
May 25 13:05:36.897131 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.897025    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zjh8s" event={"ID":"03bb8722-1d72-4446-8a56-7d37772243a4","Type":"ContainerStarted","Data":"19c6f061224e019c64f2408d99d5bdeb717629ccf9b38e3c7ea005ae500ab33d"}
May 25 13:05:36.913750 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.913705    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lvfw" event={"ID":"913e5f01-6ff3-4bd5-965e-809823df7d6d","Type":"ContainerStarted","Data":"5bdbebe247ff679b61d7747d38875c52f236f010bf00a0cac0fba9999745296e"}
May 25 13:05:36.916383 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:36.916333    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v9xkm" event={"ID":"1a62b5a7-d7eb-4389-a105-e8b1e06b1604","Type":"ContainerStarted","Data":"0028ea111a58e6896550770d97aa8a19134d59951c16876a0c83fa9fb9857ffb"}
May 25 13:05:37.358061 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:37.357361    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs\") pod \"network-metrics-daemon-tx5hr\" (UID: \"fde8fc4f-07bb-442f-aa5e-6e63efece856\") " pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:37.358061 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:37.357545    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:05:37.358061 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:37.357610    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs podName:fde8fc4f-07bb-442f-aa5e-6e63efece856 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:39.357590516 +0000 UTC m=+6.076553683 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs") pod "network-metrics-daemon-tx5hr" (UID: "fde8fc4f-07bb-442f-aa5e-6e63efece856") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:05:37.458814 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:37.458769    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k99qx\" (UniqueName: \"kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx\") pod \"network-check-target-z7xmf\" (UID: \"d4c3716c-3120-42b7-a927-bca86e4b5383\") " pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:37.459004 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:37.458958    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 13:05:37.459004 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:37.458976    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 13:05:37.459004 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:37.458989    2559 projected.go:194] Error preparing data for projected volume kube-api-access-k99qx for pod openshift-network-diagnostics/network-check-target-z7xmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:05:37.459161 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:37.459046    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx podName:d4c3716c-3120-42b7-a927-bca86e4b5383 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:39.459029697 +0000 UTC m=+6.177992822 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k99qx" (UniqueName: "kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx") pod "network-check-target-z7xmf" (UID: "d4c3716c-3120-42b7-a927-bca86e4b5383") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:05:37.867596 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:37.867548    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:37.868136 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:37.867669    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:05:37.868136 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:37.868084    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:37.868296 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:37.868180    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:05:37.944560 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:37.944526    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal" event={"ID":"8bc72641362eb99e2360a0badc226d4a","Type":"ContainerStarted","Data":"87d14e356206e61bd0b6607dcb8928c16ee0fca453ddabb34905dbd916c47353"}
May 25 13:05:37.956954 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:37.956900    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-151.ec2.internal" podStartSLOduration=3.956881894 podStartE2EDuration="3.956881894s" podCreationTimestamp="2026-05-25 13:05:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:05:37.956600986 +0000 UTC m=+4.675564126" watchObservedRunningTime="2026-05-25 13:05:37.956881894 +0000 UTC m=+4.675845036"
May 25 13:05:37.957811 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:37.957745    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-151.ec2.internal" podStartSLOduration=3.957712952 podStartE2EDuration="3.957712952s" podCreationTimestamp="2026-05-25 13:05:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:05:36.91829434 +0000 UTC m=+3.637257483" watchObservedRunningTime="2026-05-25 13:05:37.957712952 +0000 UTC m=+4.676676092"
May 25 13:05:39.373747 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:39.373217    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs\") pod \"network-metrics-daemon-tx5hr\" (UID: \"fde8fc4f-07bb-442f-aa5e-6e63efece856\") " pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:39.373747 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:39.373343    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:05:39.373747 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:39.373400    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs podName:fde8fc4f-07bb-442f-aa5e-6e63efece856 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:43.373381819 +0000 UTC m=+10.092344940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs") pod "network-metrics-daemon-tx5hr" (UID: "fde8fc4f-07bb-442f-aa5e-6e63efece856") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:05:39.474251 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:39.474217    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k99qx\" (UniqueName: \"kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx\") pod \"network-check-target-z7xmf\" (UID: \"d4c3716c-3120-42b7-a927-bca86e4b5383\") " pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:39.474434 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:39.474395    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 13:05:39.474434 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:39.474416    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 13:05:39.474434 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:39.474428    2559 projected.go:194] Error preparing data for projected volume kube-api-access-k99qx for pod openshift-network-diagnostics/network-check-target-z7xmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:05:39.474583 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:39.474490    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx podName:d4c3716c-3120-42b7-a927-bca86e4b5383 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:43.474471331 +0000 UTC m=+10.193434461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k99qx" (UniqueName: "kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx") pod "network-check-target-z7xmf" (UID: "d4c3716c-3120-42b7-a927-bca86e4b5383") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:05:39.868829 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:39.868101    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:39.868829 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:39.868239    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:05:39.868829 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:39.868608    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:39.868829 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:39.868701    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:05:41.865067 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:41.865030    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:41.865502 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:41.865082    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:41.865502 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:41.865177    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:05:41.865502 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:41.865311    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:05:43.404893 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:43.404855    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs\") pod \"network-metrics-daemon-tx5hr\" (UID: \"fde8fc4f-07bb-442f-aa5e-6e63efece856\") " pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:43.405347 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:43.405035    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:05:43.405347 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:43.405114    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs podName:fde8fc4f-07bb-442f-aa5e-6e63efece856 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:51.40509196 +0000 UTC m=+18.124055100 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs") pod "network-metrics-daemon-tx5hr" (UID: "fde8fc4f-07bb-442f-aa5e-6e63efece856") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:05:43.506004 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:43.505961    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k99qx\" (UniqueName: \"kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx\") pod \"network-check-target-z7xmf\" (UID: \"d4c3716c-3120-42b7-a927-bca86e4b5383\") " pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:43.506189 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:43.506150    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 13:05:43.506189 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:43.506173    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 13:05:43.506189 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:43.506186    2559 projected.go:194] Error preparing data for projected volume kube-api-access-k99qx for pod openshift-network-diagnostics/network-check-target-z7xmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:05:43.506340 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:43.506253    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx podName:d4c3716c-3120-42b7-a927-bca86e4b5383 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:51.506235684 +0000 UTC m=+18.225198813 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k99qx" (UniqueName: "kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx") pod "network-check-target-z7xmf" (UID: "d4c3716c-3120-42b7-a927-bca86e4b5383") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:05:43.865986 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:43.865364    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:43.865986 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:43.865467    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:05:43.865986 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:43.865845    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:43.865986 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:43.865945    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:05:45.868078 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:45.868051    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:45.868529 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:45.868052    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:45.868529 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:45.868168    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:05:45.868529 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:45.868263    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:05:47.867758 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:47.867713    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:47.868201 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:47.867713    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:47.868201 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:47.867843    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:05:47.868201 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:47.867886    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:05:48.327797 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:48.327704    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-xfbms"]
May 25 13:05:48.330553 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:48.330531    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:48.330671 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:48.330613    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfbms" podUID="fbb8062f-fbfd-4af1-ae8d-c7fb90612985"
May 25 13:05:48.441934 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:48.441892    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-kubelet-config\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:48.441934 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:48.441925    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-dbus\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:48.442141 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:48.441959    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:48.542848 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:48.542807    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:48.543011 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:48.542900    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-kubelet-config\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:48.543011 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:48.542921    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-dbus\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:48.543011 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:48.542972    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 25 13:05:48.543189 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:48.543048    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret podName:fbb8062f-fbfd-4af1-ae8d-c7fb90612985 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:49.043025207 +0000 UTC m=+15.761988329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret") pod "global-pull-secret-syncer-xfbms" (UID: "fbb8062f-fbfd-4af1-ae8d-c7fb90612985") : object "kube-system"/"original-pull-secret" not registered
May 25 13:05:48.543189 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:48.543051    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-kubelet-config\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:48.543189 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:48.543070    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-dbus\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:49.046983 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:49.046948    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:49.047397 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:49.047111    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 25 13:05:49.047397 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:49.047185    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret podName:fbb8062f-fbfd-4af1-ae8d-c7fb90612985 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:50.047165954 +0000 UTC m=+16.766129087 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret") pod "global-pull-secret-syncer-xfbms" (UID: "fbb8062f-fbfd-4af1-ae8d-c7fb90612985") : object "kube-system"/"original-pull-secret" not registered
May 25 13:05:49.864509 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:49.864481    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:49.864509 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:49.864499    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:49.864764 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:49.864481    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:49.864764 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:49.864595    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:05:49.864764 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:49.864683    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:05:49.864903 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:49.864785    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfbms" podUID="fbb8062f-fbfd-4af1-ae8d-c7fb90612985"
May 25 13:05:50.054259 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:50.054217    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:50.054669 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:50.054363    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 25 13:05:50.054669 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:50.054433    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret podName:fbb8062f-fbfd-4af1-ae8d-c7fb90612985 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:52.054411923 +0000 UTC m=+18.773375053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret") pod "global-pull-secret-syncer-xfbms" (UID: "fbb8062f-fbfd-4af1-ae8d-c7fb90612985") : object "kube-system"/"original-pull-secret" not registered
May 25 13:05:51.464868 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:51.464833    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs\") pod \"network-metrics-daemon-tx5hr\" (UID: \"fde8fc4f-07bb-442f-aa5e-6e63efece856\") " pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:51.465341 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:51.465007    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:05:51.465341 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:51.465091    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs podName:fde8fc4f-07bb-442f-aa5e-6e63efece856 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:07.465068326 +0000 UTC m=+34.184031457 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs") pod "network-metrics-daemon-tx5hr" (UID: "fde8fc4f-07bb-442f-aa5e-6e63efece856") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:05:51.565677 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:51.565635    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k99qx\" (UniqueName: \"kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx\") pod \"network-check-target-z7xmf\" (UID: \"d4c3716c-3120-42b7-a927-bca86e4b5383\") " pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:51.565933 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:51.565840    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 13:05:51.565933 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:51.565866    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 13:05:51.565933 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:51.565881    2559 projected.go:194] Error preparing data for projected volume kube-api-access-k99qx for pod openshift-network-diagnostics/network-check-target-z7xmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:05:51.566093 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:51.565946    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx podName:d4c3716c-3120-42b7-a927-bca86e4b5383 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:07.56593133 +0000 UTC m=+34.284894587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k99qx" (UniqueName: "kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx") pod "network-check-target-z7xmf" (UID: "d4c3716c-3120-42b7-a927-bca86e4b5383") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:05:51.864414 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:51.864324    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:51.864582 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:51.864324    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:51.864582 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:51.864450    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfbms" podUID="fbb8062f-fbfd-4af1-ae8d-c7fb90612985"
May 25 13:05:51.864582 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:51.864324    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:51.864582 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:51.864546    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:05:51.864812 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:51.864598    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:05:52.068970 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:52.068934    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:52.069151 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:52.069052    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 25 13:05:52.069151 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:52.069112    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret podName:fbb8062f-fbfd-4af1-ae8d-c7fb90612985 nodeName:}" failed. No retries permitted until 2026-05-25 13:05:56.069093949 +0000 UTC m=+22.788057065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret") pod "global-pull-secret-syncer-xfbms" (UID: "fbb8062f-fbfd-4af1-ae8d-c7fb90612985") : object "kube-system"/"original-pull-secret" not registered
May 25 13:05:53.865715 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:53.865686    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:53.866065 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:53.865851    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:53.866065 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:53.865851    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfbms" podUID="fbb8062f-fbfd-4af1-ae8d-c7fb90612985"
May 25 13:05:53.866065 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:53.865954    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:05:53.866065 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:53.866006    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:53.866191 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:53.866094    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:05:54.987519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:54.987293    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs" event={"ID":"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35","Type":"ContainerStarted","Data":"02a1d41c590f1b13d139d47fc414a76907c9fc08ce14e4744fddb28494de602f"}
May 25 13:05:54.987519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:54.987488    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs" event={"ID":"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35","Type":"ContainerStarted","Data":"f0bd881de986d1d70be8024c3e928c574ec8385ff0110c98719934b2f5027fad"}
May 25 13:05:54.987519 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:54.987509    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs" event={"ID":"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35","Type":"ContainerStarted","Data":"cf1089b095cfec636b2ce7c85803802f5be62bf7519d3577e533a14da70b960b"}
May 25 13:05:54.988376 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:54.987522    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs" event={"ID":"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35","Type":"ContainerStarted","Data":"1b836c248be54f075ee60f20cc05fc12835387817fb3d7c0fa45c73f1df0974e"}
May 25 13:05:54.988376 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:54.987536    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs" event={"ID":"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35","Type":"ContainerStarted","Data":"e52d398c5f4abdfd85f03695bfccfe27c5dc617e5119467224623b9a950f92b1"}
May 25 13:05:54.988376 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:54.987548    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs" event={"ID":"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35","Type":"ContainerStarted","Data":"da1fe292a631770c20ff54b568b246bd4331f3a4dd82a9c25cabce22bb296210"}
May 25 13:05:54.988582 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:54.988558    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tghhw" event={"ID":"c886a9b9-c5da-4569-a695-125c80d706f6","Type":"ContainerStarted","Data":"13bfe5a085306ef93f93f3939b4afe245f85c3be3bd229915b55fb9bf3394b9e"}
May 25 13:05:54.989803 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:54.989775    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m6244" event={"ID":"54e5ea0f-6319-42c2-9770-76a93d975631","Type":"ContainerStarted","Data":"202c7b137d3c9b04ed6da63fe2caaa104e5c41c3a7fa9ebfde36ce06f579cc34"}
May 25 13:05:54.991040 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:54.991021    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv" event={"ID":"b2ceae0e-518e-4e3c-9892-d50a78b74642","Type":"ContainerStarted","Data":"b2365628d99b95250a4c2dba131a18b2c098351f993075435da3d73e4b6b5557"}
May 25 13:05:54.992310 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:54.992289    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zjh8s" event={"ID":"03bb8722-1d72-4446-8a56-7d37772243a4","Type":"ContainerStarted","Data":"d22ca4163f5526efb742fbfc3bed5e9b4ea2e2a317cab187b9ffd4370dae2680"}
May 25 13:05:54.993610 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:54.993590    2559 generic.go:358] "Generic (PLEG): container finished" podID="913e5f01-6ff3-4bd5-965e-809823df7d6d" containerID="3ab596e6ed5f08973528360dfc5a3b1eed2552fae5c7b177683d342b6c2ac85e" exitCode=0
May 25 13:05:54.993712 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:54.993653    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lvfw" event={"ID":"913e5f01-6ff3-4bd5-965e-809823df7d6d","Type":"ContainerDied","Data":"3ab596e6ed5f08973528360dfc5a3b1eed2552fae5c7b177683d342b6c2ac85e"}
May 25 13:05:54.994920 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:54.994901    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-95pwd" event={"ID":"bc716b46-7c31-42f5-ba6f-45b8c77d29ad","Type":"ContainerStarted","Data":"eebd62a93f4cce74995bf43e86113586f5723e45f8e6119deb3576c44b77faaa"}
May 25 13:05:55.004924 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.002854    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-tghhw" podStartSLOduration=4.457203279 podStartE2EDuration="22.002822356s" podCreationTimestamp="2026-05-25 13:05:33 +0000 UTC" firstStartedPulling="2026-05-25 13:05:36.414461368 +0000 UTC m=+3.133424489" lastFinishedPulling="2026-05-25 13:05:53.960080445 +0000 UTC m=+20.679043566" observedRunningTime="2026-05-25 13:05:55.002709771 +0000 UTC m=+21.721672914" watchObservedRunningTime="2026-05-25 13:05:55.002822356 +0000 UTC m=+21.721785494"
May 25 13:05:55.027960 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.027771    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-95pwd" podStartSLOduration=8.210878283 podStartE2EDuration="21.027761432s" podCreationTimestamp="2026-05-25 13:05:34 +0000 UTC" firstStartedPulling="2026-05-25 13:05:36.407212656 +0000 UTC m=+3.126175788" lastFinishedPulling="2026-05-25 13:05:49.224095807 +0000 UTC m=+15.943058937" observedRunningTime="2026-05-25 13:05:55.027445716 +0000 UTC m=+21.746408869" watchObservedRunningTime="2026-05-25 13:05:55.027761432 +0000 UTC m=+21.746724570"
May 25 13:05:55.038326 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.038288    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zjh8s" podStartSLOduration=4.5497174000000005 podStartE2EDuration="22.038277542s" podCreationTimestamp="2026-05-25 13:05:33 +0000 UTC" firstStartedPulling="2026-05-25 13:05:36.415539683 +0000 UTC m=+3.134502811" lastFinishedPulling="2026-05-25 13:05:53.904099822 +0000 UTC m=+20.623062953" observedRunningTime="2026-05-25 13:05:55.038191598 +0000 UTC m=+21.757154738" watchObservedRunningTime="2026-05-25 13:05:55.038277542 +0000 UTC m=+21.757240692"
May 25 13:05:55.114286 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.114264    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 25 13:05:55.673156 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.673079    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zjh8s"
May 25 13:05:55.674255 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.674225    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zjh8s"
May 25 13:05:55.685955 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.685815    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m6244" podStartSLOduration=4.149799271 podStartE2EDuration="21.68579755s" podCreationTimestamp="2026-05-25 13:05:34 +0000 UTC" firstStartedPulling="2026-05-25 13:05:36.412849567 +0000 UTC m=+3.131812699" lastFinishedPulling="2026-05-25 13:05:53.948847853 +0000 UTC m=+20.667810978" observedRunningTime="2026-05-25 13:05:55.05039504 +0000 UTC m=+21.769358179" watchObservedRunningTime="2026-05-25 13:05:55.68579755 +0000 UTC m=+22.404760691"
May 25 13:05:55.802174 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.802050    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-25T13:05:55.114282169Z","UUID":"97bee633-b91e-4016-9490-675074a854ef","Handler":null,"Name":"","Endpoint":""}
May 25 13:05:55.804007 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.803984    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 25 13:05:55.804007 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.804017    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 25 13:05:55.864844 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.864792    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:55.865021 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.864796    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:55.865021 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:55.864903    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:05:55.865021 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:55.865000    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfbms" podUID="fbb8062f-fbfd-4af1-ae8d-c7fb90612985"
May 25 13:05:55.865185 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.865047    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:55.865185 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:55.865122    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:05:55.998769 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:55.998733    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v9xkm" event={"ID":"1a62b5a7-d7eb-4389-a105-e8b1e06b1604","Type":"ContainerStarted","Data":"cda094c36b267366df331ec023b9811283b19e1be7c7f323bfff73829438c2f8"}
May 25 13:05:56.000984 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:56.000486    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv" event={"ID":"b2ceae0e-518e-4e3c-9892-d50a78b74642","Type":"ContainerStarted","Data":"005feeba3bf841481b0a674bccc0700d7ed3e3161c0ad295eefb8e75f4383002"}
May 25 13:05:56.024649 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:56.024613    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-v9xkm" podStartSLOduration=5.486288848 podStartE2EDuration="23.02460128s" podCreationTimestamp="2026-05-25 13:05:33 +0000 UTC" firstStartedPulling="2026-05-25 13:05:36.409108854 +0000 UTC m=+3.128071975" lastFinishedPulling="2026-05-25 13:05:53.947421275 +0000 UTC m=+20.666384407" observedRunningTime="2026-05-25 13:05:56.023772161 +0000 UTC m=+22.742735301" watchObservedRunningTime="2026-05-25 13:05:56.02460128 +0000 UTC m=+22.743564420"
May 25 13:05:56.093466 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:56.093329    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:56.094386 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:56.094361    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 25 13:05:56.094490 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:56.094436    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret podName:fbb8062f-fbfd-4af1-ae8d-c7fb90612985 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:04.09441562 +0000 UTC m=+30.813378752 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret") pod "global-pull-secret-syncer-xfbms" (UID: "fbb8062f-fbfd-4af1-ae8d-c7fb90612985") : object "kube-system"/"original-pull-secret" not registered
May 25 13:05:57.004269 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:57.004191    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv" event={"ID":"b2ceae0e-518e-4e3c-9892-d50a78b74642","Type":"ContainerStarted","Data":"d5c24fa29a145aec2d3cd86186301dd3927f3cc49bac054cd2ef99bcda7d355d"}
May 25 13:05:57.007059 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:57.007033    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs" event={"ID":"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35","Type":"ContainerStarted","Data":"35bbdc397a86e93e5e5298bf29099ecb874db3d2e0d7d5327a135d087428a871"}
May 25 13:05:57.007207 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:57.007190    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 25 13:05:57.021342 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:57.021304    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-76tgv" podStartSLOduration=4.470760365 podStartE2EDuration="24.021294656s" podCreationTimestamp="2026-05-25 13:05:33 +0000 UTC" firstStartedPulling="2026-05-25 13:05:36.41412629 +0000 UTC m=+3.133089407" lastFinishedPulling="2026-05-25 13:05:55.964660544 +0000 UTC m=+22.683623698" observedRunningTime="2026-05-25 13:05:57.020830389 +0000 UTC m=+23.739793547" watchObservedRunningTime="2026-05-25 13:05:57.021294656 +0000 UTC m=+23.740257794"
May 25 13:05:57.865381 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:57.865349    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:57.865542 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:57.865349    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:57.865542 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:57.865496    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:05:57.865542 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:57.865350    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:57.865694 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:57.865629    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:05:57.865756 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:57.865738    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfbms" podUID="fbb8062f-fbfd-4af1-ae8d-c7fb90612985"
May 25 13:05:59.865127 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:59.864411    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:05:59.865127 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:59.864414    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:05:59.865127 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:59.864746    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfbms" podUID="fbb8062f-fbfd-4af1-ae8d-c7fb90612985"
May 25 13:05:59.865127 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:59.864822    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:05:59.865127 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:05:59.864472    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:05:59.865127 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:05:59.864905    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:06:00.013739 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:00.013693    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lvfw" event={"ID":"913e5f01-6ff3-4bd5-965e-809823df7d6d","Type":"ContainerStarted","Data":"bbc748b53312658d78c7bcba72081b22bda541644b6212938f8f464c0f5327f4"}
May 25 13:06:00.016804 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:00.016778    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs" event={"ID":"724bf4c7-1d41-4ddf-aa5e-d8b043c08c35","Type":"ContainerStarted","Data":"23c5f701ac1075bdd32927d174f971b542e7c8be42eb143690e6eb4c71574bd1"}
May 25 13:06:00.017335 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:00.017316    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:06:00.017394 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:00.017345    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:06:00.031979 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:00.031896    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:06:00.031979 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:00.031961    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:06:00.044524 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:00.044466    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs" podStartSLOduration=9.394437565 podStartE2EDuration="27.044447897s" podCreationTimestamp="2026-05-25 13:05:33 +0000 UTC" firstStartedPulling="2026-05-25 13:05:36.405734058 +0000 UTC m=+3.124697178" lastFinishedPulling="2026-05-25 13:05:54.055744381 +0000 UTC m=+20.774707510" observedRunningTime="2026-05-25 13:06:00.043989823 +0000 UTC m=+26.762952963" watchObservedRunningTime="2026-05-25 13:06:00.044447897 +0000 UTC m=+26.763411037"
May 25 13:06:01.019869 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:01.019836    2559 generic.go:358] "Generic (PLEG): container finished" podID="913e5f01-6ff3-4bd5-965e-809823df7d6d" containerID="bbc748b53312658d78c7bcba72081b22bda541644b6212938f8f464c0f5327f4" exitCode=0
May 25 13:06:01.020241 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:01.019926    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lvfw" event={"ID":"913e5f01-6ff3-4bd5-965e-809823df7d6d","Type":"ContainerDied","Data":"bbc748b53312658d78c7bcba72081b22bda541644b6212938f8f464c0f5327f4"}
May 25 13:06:01.020241 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:01.020104    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 25 13:06:01.665615 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:01.665579    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z7xmf"]
May 25 13:06:01.665819 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:01.665735    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:06:01.665881 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:01.665852    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:06:01.666261 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:01.666213    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xfbms"]
May 25 13:06:01.666367 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:01.666331    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:06:01.666444 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:01.666425    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfbms" podUID="fbb8062f-fbfd-4af1-ae8d-c7fb90612985"
May 25 13:06:01.666774 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:01.666759    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tx5hr"]
May 25 13:06:01.666856 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:01.666845    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:06:01.666927 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:01.666914    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:06:02.022037 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:02.022007    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 25 13:06:02.865105 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:02.865069    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:06:02.865295 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:02.865190    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfbms" podUID="fbb8062f-fbfd-4af1-ae8d-c7fb90612985"
May 25 13:06:02.894641 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:02.894608    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zjh8s"
May 25 13:06:02.894834 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:02.894776    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 25 13:06:02.895286 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:02.895264    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zjh8s"
May 25 13:06:03.025373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:03.025339    2559 generic.go:358] "Generic (PLEG): container finished" podID="913e5f01-6ff3-4bd5-965e-809823df7d6d" containerID="4915d535f6025fa13b836ff8682dedc71b0f75c5d9613f35abbc2564fdc79852" exitCode=0
May 25 13:06:03.025857 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:03.025426    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lvfw" event={"ID":"913e5f01-6ff3-4bd5-965e-809823df7d6d","Type":"ContainerDied","Data":"4915d535f6025fa13b836ff8682dedc71b0f75c5d9613f35abbc2564fdc79852"}
May 25 13:06:03.865912 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:03.865892    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:06:03.866029 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:03.865964    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:06:03.866080 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:03.866047    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:06:03.866151 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:03.866134    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:06:04.029199 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:04.029117    2559 generic.go:358] "Generic (PLEG): container finished" podID="913e5f01-6ff3-4bd5-965e-809823df7d6d" containerID="9c0d1d1c6510f43b9f1232a1fec8cc645fc39778937546b6edbbf255f2ac30f8" exitCode=0
May 25 13:06:04.029636 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:04.029209    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lvfw" event={"ID":"913e5f01-6ff3-4bd5-965e-809823df7d6d","Type":"ContainerDied","Data":"9c0d1d1c6510f43b9f1232a1fec8cc645fc39778937546b6edbbf255f2ac30f8"}
May 25 13:06:04.152601 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:04.152568    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:06:04.152839 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:04.152822    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 25 13:06:04.152916 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:04.152897    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret podName:fbb8062f-fbfd-4af1-ae8d-c7fb90612985 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:20.152875256 +0000 UTC m=+46.871838379 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret") pod "global-pull-secret-syncer-xfbms" (UID: "fbb8062f-fbfd-4af1-ae8d-c7fb90612985") : object "kube-system"/"original-pull-secret" not registered
May 25 13:06:04.799023 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:04.798992    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:06:04.799242 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:04.799225    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 25 13:06:04.816521 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:04.816466    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs" podUID="724bf4c7-1d41-4ddf-aa5e-d8b043c08c35" containerName="ovnkube-controller" probeResult="failure" output=""
May 25 13:06:04.825841 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:04.825807    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs" podUID="724bf4c7-1d41-4ddf-aa5e-d8b043c08c35" containerName="ovnkube-controller" probeResult="failure" output=""
May 25 13:06:04.864653 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:04.864631    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:06:04.864787 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:04.864766    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xfbms" podUID="fbb8062f-fbfd-4af1-ae8d-c7fb90612985"
May 25 13:06:05.864621 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:05.864573    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:06:05.865095 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:05.864589    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:06:05.865095 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:05.864729    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tx5hr" podUID="fde8fc4f-07bb-442f-aa5e-6e63efece856"
May 25 13:06:05.865095 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:05.864806    2559 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7xmf" podUID="d4c3716c-3120-42b7-a927-bca86e4b5383"
May 25 13:06:06.596017 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.595990    2559 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-129-151.ec2.internal" event="NodeReady"
May 25 13:06:06.596180 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.596119    2559 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 25 13:06:06.644771 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.644734    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-845c4b7766-sckdj"]
May 25 13:06:06.651449 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.651424    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.654404 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.654380    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 25 13:06:06.654658 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.654641    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 25 13:06:06.654950 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.654886    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 25 13:06:06.655620 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.655595    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lntzn\""
May 25 13:06:06.663959 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.663433    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lftth"]
May 25 13:06:06.665998 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.665974    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 25 13:06:06.671302 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.671280    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-845c4b7766-sckdj"]
May 25 13:06:06.671414 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.671400    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:06.674545 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.673286    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lftth"]
May 25 13:06:06.674545 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.673977    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 25 13:06:06.674545 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.674235    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 25 13:06:06.674545 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.674434    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vbzlk\""
May 25 13:06:06.674791 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.674605    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 25 13:06:06.679379 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.679358    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f6lwr"]
May 25 13:06:06.689595 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.689575    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:06.692520 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.692494    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 25 13:06:06.693280 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.692670    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5d4pm\""
May 25 13:06:06.693280 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.693012    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 25 13:06:06.693280 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.693074    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 25 13:06:06.693280 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.693184    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 25 13:06:06.694066 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.694044    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f6lwr"]
May 25 13:06:06.769581 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.769551    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vxxn\" (UniqueName: \"kubernetes.io/projected/7844394a-a353-4e6f-9c29-75e111adcdb9-kube-api-access-8vxxn\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:06.769787 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.769612    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.769787 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.769672    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-certificates\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.769787 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.769710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-image-registry-private-configuration\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.769958 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.769792    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert\") pod \"ingress-canary-lftth\" (UID: \"95c09ce4-ed3a-4de9-baef-5b4e1fed5652\") " pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:06.769958 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.769830    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-installation-pull-secrets\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.769958 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.769879    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7844394a-a353-4e6f-9c29-75e111adcdb9-config-volume\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:06.769958 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.769902    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxljm\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-kube-api-access-lxljm\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.769958 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.769931    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:06.769958 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.769957    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-bound-sa-token\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.770219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.769994    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-ca-trust-extracted\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.770219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.770025    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d5bv\" (UniqueName: \"kubernetes.io/projected/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-kube-api-access-8d5bv\") pod \"ingress-canary-lftth\" (UID: \"95c09ce4-ed3a-4de9-baef-5b4e1fed5652\") " pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:06.770219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.770048    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7844394a-a353-4e6f-9c29-75e111adcdb9-tmp-dir\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:06.770219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.770074    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-trusted-ca\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.864609 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.864575    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:06:06.867155 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.867133    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 25 13:06:06.870731 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.870684    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8vxxn\" (UniqueName: \"kubernetes.io/projected/7844394a-a353-4e6f-9c29-75e111adcdb9-kube-api-access-8vxxn\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:06.870850 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.870751    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.870850 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.870816    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-certificates\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.871026 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.870879    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-image-registry-private-configuration\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.871026 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:06.870898    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 13:06:06.871026 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.870915    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert\") pod \"ingress-canary-lftth\" (UID: \"95c09ce4-ed3a-4de9-baef-5b4e1fed5652\") " pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:06.871026 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.870949    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-installation-pull-secrets\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.871026 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.870995    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7844394a-a353-4e6f-9c29-75e111adcdb9-config-volume\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:06.871026 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.871022    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lxljm\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-kube-api-access-lxljm\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.871308 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.871060    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:06.871308 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:06.870918    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-845c4b7766-sckdj: secret "image-registry-tls" not found
May 25 13:06:06.871308 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:06.871169    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls podName:05ac39e7-0786-4cd3-9b66-05091c2c0c1a nodeName:}" failed. No retries permitted until 2026-05-25 13:06:07.371148138 +0000 UTC m=+34.090111259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls") pod "image-registry-845c4b7766-sckdj" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a") : secret "image-registry-tls" not found
May 25 13:06:06.871308 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.871100    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-bound-sa-token\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.871308 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:06.871056    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 13:06:06.871560 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.871312    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-ca-trust-extracted\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.871560 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:06.871337    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert podName:95c09ce4-ed3a-4de9-baef-5b4e1fed5652 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:07.371318742 +0000 UTC m=+34.090281867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert") pod "ingress-canary-lftth" (UID: "95c09ce4-ed3a-4de9-baef-5b4e1fed5652") : secret "canary-serving-cert" not found
May 25 13:06:06.871560 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.871377    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8d5bv\" (UniqueName: \"kubernetes.io/projected/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-kube-api-access-8d5bv\") pod \"ingress-canary-lftth\" (UID: \"95c09ce4-ed3a-4de9-baef-5b4e1fed5652\") " pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:06.871560 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:06.871460    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 13:06:06.871560 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:06.871509    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls podName:7844394a-a353-4e6f-9c29-75e111adcdb9 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:07.371493679 +0000 UTC m=+34.090456799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls") pod "dns-default-f6lwr" (UID: "7844394a-a353-4e6f-9c29-75e111adcdb9") : secret "dns-default-metrics-tls" not found
May 25 13:06:06.871851 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.871605    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7844394a-a353-4e6f-9c29-75e111adcdb9-config-volume\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:06.871851 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.871611    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7844394a-a353-4e6f-9c29-75e111adcdb9-tmp-dir\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:06.871851 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.871646    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-trusted-ca\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.871851 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.871680    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-ca-trust-extracted\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.871851 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.871680    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-certificates\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.872091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.871884    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7844394a-a353-4e6f-9c29-75e111adcdb9-tmp-dir\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:06.873199 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.872909    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-trusted-ca\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.875845 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.875824    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-image-registry-private-configuration\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.875965 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.875846    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-installation-pull-secrets\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.880137 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.880082    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-bound-sa-token\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.880137 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.880089    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxljm\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-kube-api-access-lxljm\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:06.880137 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.880117    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vxxn\" (UniqueName: \"kubernetes.io/projected/7844394a-a353-4e6f-9c29-75e111adcdb9-kube-api-access-8vxxn\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:06.880426 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:06.880405    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d5bv\" (UniqueName: \"kubernetes.io/projected/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-kube-api-access-8d5bv\") pod \"ingress-canary-lftth\" (UID: \"95c09ce4-ed3a-4de9-baef-5b4e1fed5652\") " pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:07.043031 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.042955    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-482g6"]
May 25 13:06:07.073032 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.073004    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-482g6"
May 25 13:06:07.075248 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.075228    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wmpmm\""
May 25 13:06:07.173806 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.173771    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9ea8b5d3-ab26-4393-b5f9-747d95405510-hosts-file\") pod \"node-resolver-482g6\" (UID: \"9ea8b5d3-ab26-4393-b5f9-747d95405510\") " pod="openshift-dns/node-resolver-482g6"
May 25 13:06:07.173995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.173817    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbmcd\" (UniqueName: \"kubernetes.io/projected/9ea8b5d3-ab26-4393-b5f9-747d95405510-kube-api-access-sbmcd\") pod \"node-resolver-482g6\" (UID: \"9ea8b5d3-ab26-4393-b5f9-747d95405510\") " pod="openshift-dns/node-resolver-482g6"
May 25 13:06:07.173995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.173894    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ea8b5d3-ab26-4393-b5f9-747d95405510-tmp-dir\") pod \"node-resolver-482g6\" (UID: \"9ea8b5d3-ab26-4393-b5f9-747d95405510\") " pod="openshift-dns/node-resolver-482g6"
May 25 13:06:07.275079 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.275043    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9ea8b5d3-ab26-4393-b5f9-747d95405510-hosts-file\") pod \"node-resolver-482g6\" (UID: \"9ea8b5d3-ab26-4393-b5f9-747d95405510\") " pod="openshift-dns/node-resolver-482g6"
May 25 13:06:07.275079 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.275085    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sbmcd\" (UniqueName: \"kubernetes.io/projected/9ea8b5d3-ab26-4393-b5f9-747d95405510-kube-api-access-sbmcd\") pod \"node-resolver-482g6\" (UID: \"9ea8b5d3-ab26-4393-b5f9-747d95405510\") " pod="openshift-dns/node-resolver-482g6"
May 25 13:06:07.275324 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.275113    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ea8b5d3-ab26-4393-b5f9-747d95405510-tmp-dir\") pod \"node-resolver-482g6\" (UID: \"9ea8b5d3-ab26-4393-b5f9-747d95405510\") " pod="openshift-dns/node-resolver-482g6"
May 25 13:06:07.275324 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.275184    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9ea8b5d3-ab26-4393-b5f9-747d95405510-hosts-file\") pod \"node-resolver-482g6\" (UID: \"9ea8b5d3-ab26-4393-b5f9-747d95405510\") " pod="openshift-dns/node-resolver-482g6"
May 25 13:06:07.275778 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.275757    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ea8b5d3-ab26-4393-b5f9-747d95405510-tmp-dir\") pod \"node-resolver-482g6\" (UID: \"9ea8b5d3-ab26-4393-b5f9-747d95405510\") " pod="openshift-dns/node-resolver-482g6"
May 25 13:06:07.282815 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.282789    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbmcd\" (UniqueName: \"kubernetes.io/projected/9ea8b5d3-ab26-4393-b5f9-747d95405510-kube-api-access-sbmcd\") pod \"node-resolver-482g6\" (UID: \"9ea8b5d3-ab26-4393-b5f9-747d95405510\") " pod="openshift-dns/node-resolver-482g6"
May 25 13:06:07.376197 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.376114    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert\") pod \"ingress-canary-lftth\" (UID: \"95c09ce4-ed3a-4de9-baef-5b4e1fed5652\") " pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:07.376197 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.376184    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:07.376407 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.376247    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:07.376407 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:07.376261    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 13:06:07.376407 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:07.376321    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert podName:95c09ce4-ed3a-4de9-baef-5b4e1fed5652 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:08.376302822 +0000 UTC m=+35.095265938 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert") pod "ingress-canary-lftth" (UID: "95c09ce4-ed3a-4de9-baef-5b4e1fed5652") : secret "canary-serving-cert" not found
May 25 13:06:07.376407 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:07.376324    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 13:06:07.376407 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:07.376339    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 13:06:07.376407 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:07.376351    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-845c4b7766-sckdj: secret "image-registry-tls" not found
May 25 13:06:07.376407 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:07.376383    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls podName:7844394a-a353-4e6f-9c29-75e111adcdb9 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:08.376367486 +0000 UTC m=+35.095330607 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls") pod "dns-default-f6lwr" (UID: "7844394a-a353-4e6f-9c29-75e111adcdb9") : secret "dns-default-metrics-tls" not found
May 25 13:06:07.376407 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:07.376398    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls podName:05ac39e7-0786-4cd3-9b66-05091c2c0c1a nodeName:}" failed. No retries permitted until 2026-05-25 13:06:08.376390965 +0000 UTC m=+35.095354082 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls") pod "image-registry-845c4b7766-sckdj" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a") : secret "image-registry-tls" not found
May 25 13:06:07.383567 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.383551    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-482g6"
May 25 13:06:07.392416 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:07.392385    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ea8b5d3_ab26_4393_b5f9_747d95405510.slice/crio-11e0e30cb2b2e9c9488ae885f2b79d4a474e0c00f71c9fae7155833af9270422 WatchSource:0}: Error finding container 11e0e30cb2b2e9c9488ae885f2b79d4a474e0c00f71c9fae7155833af9270422: Status 404 returned error can't find the container with id 11e0e30cb2b2e9c9488ae885f2b79d4a474e0c00f71c9fae7155833af9270422
May 25 13:06:07.476583 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.476554    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs\") pod \"network-metrics-daemon-tx5hr\" (UID: \"fde8fc4f-07bb-442f-aa5e-6e63efece856\") " pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:06:07.476739 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:07.476685    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:06:07.476790 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:07.476756    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs podName:fde8fc4f-07bb-442f-aa5e-6e63efece856 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:39.476739889 +0000 UTC m=+66.195703006 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs") pod "network-metrics-daemon-tx5hr" (UID: "fde8fc4f-07bb-442f-aa5e-6e63efece856") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 13:06:07.577581 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.577546    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k99qx\" (UniqueName: \"kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx\") pod \"network-check-target-z7xmf\" (UID: \"d4c3716c-3120-42b7-a927-bca86e4b5383\") " pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:06:07.577806 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:07.577767    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 13:06:07.577806 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:07.577794    2559 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 13:06:07.577806 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:07.577807    2559 projected.go:194] Error preparing data for projected volume kube-api-access-k99qx for pod openshift-network-diagnostics/network-check-target-z7xmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:06:07.577974 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:07.577869    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx podName:d4c3716c-3120-42b7-a927-bca86e4b5383 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:39.577850095 +0000 UTC m=+66.296813215 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k99qx" (UniqueName: "kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx") pod "network-check-target-z7xmf" (UID: "d4c3716c-3120-42b7-a927-bca86e4b5383") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 13:06:07.864958 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.864916    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:06:07.865143 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.864972    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:06:07.869744 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.868360    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-clhmx\""
May 25 13:06:07.869744 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.868411    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 25 13:06:07.869744 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.868470    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 25 13:06:07.869744 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.869232    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 25 13:06:07.870428 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:07.870300    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gcddr\""
May 25 13:06:08.039319 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:08.039279    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-482g6" event={"ID":"9ea8b5d3-ab26-4393-b5f9-747d95405510","Type":"ContainerStarted","Data":"11e0e30cb2b2e9c9488ae885f2b79d4a474e0c00f71c9fae7155833af9270422"}
May 25 13:06:08.383949 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:08.383908    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert\") pod \"ingress-canary-lftth\" (UID: \"95c09ce4-ed3a-4de9-baef-5b4e1fed5652\") " pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:08.384149 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:08.383970    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:08.384149 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:08.384012    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:08.384149 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:08.384061    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 13:06:08.384149 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:08.384111    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 13:06:08.384149 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:08.384125    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-845c4b7766-sckdj: secret "image-registry-tls" not found
May 25 13:06:08.384149 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:08.384139    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 13:06:08.384404 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:08.384145    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert podName:95c09ce4-ed3a-4de9-baef-5b4e1fed5652 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:10.384122453 +0000 UTC m=+37.103085581 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert") pod "ingress-canary-lftth" (UID: "95c09ce4-ed3a-4de9-baef-5b4e1fed5652") : secret "canary-serving-cert" not found
May 25 13:06:08.384404 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:08.384206    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls podName:05ac39e7-0786-4cd3-9b66-05091c2c0c1a nodeName:}" failed. No retries permitted until 2026-05-25 13:06:10.384188329 +0000 UTC m=+37.103151462 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls") pod "image-registry-845c4b7766-sckdj" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a") : secret "image-registry-tls" not found
May 25 13:06:08.384404 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:08.384222    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls podName:7844394a-a353-4e6f-9c29-75e111adcdb9 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:10.384212745 +0000 UTC m=+37.103175869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls") pod "dns-default-f6lwr" (UID: "7844394a-a353-4e6f-9c29-75e111adcdb9") : secret "dns-default-metrics-tls" not found
May 25 13:06:10.046009 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:10.045973    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lvfw" event={"ID":"913e5f01-6ff3-4bd5-965e-809823df7d6d","Type":"ContainerStarted","Data":"a67a472febe21dcb4f8f01e36f7d7cd6b7e6764ddc32ae5282ba39a98891d376"}
May 25 13:06:10.047395 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:10.047369    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-482g6" event={"ID":"9ea8b5d3-ab26-4393-b5f9-747d95405510","Type":"ContainerStarted","Data":"70e2c42b5aa07ff279d01d62b4bf9044d47c95e4889ac1f17ba7eef280389902"}
May 25 13:06:10.096018 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:10.095977    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-482g6" podStartSLOduration=3.095961025 podStartE2EDuration="3.095961025s" podCreationTimestamp="2026-05-25 13:06:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:06:10.095270214 +0000 UTC m=+36.814233353" watchObservedRunningTime="2026-05-25 13:06:10.095961025 +0000 UTC m=+36.814924161"
May 25 13:06:10.398392 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:10.398305    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert\") pod \"ingress-canary-lftth\" (UID: \"95c09ce4-ed3a-4de9-baef-5b4e1fed5652\") " pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:10.398392 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:10.398363    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:10.398567 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:10.398445    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 13:06:10.398567 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:10.398449    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 13:06:10.398567 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:10.398499    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls podName:7844394a-a353-4e6f-9c29-75e111adcdb9 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:14.398483866 +0000 UTC m=+41.117446983 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls") pod "dns-default-f6lwr" (UID: "7844394a-a353-4e6f-9c29-75e111adcdb9") : secret "dns-default-metrics-tls" not found
May 25 13:06:10.398664 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:10.398571    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert podName:95c09ce4-ed3a-4de9-baef-5b4e1fed5652 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:14.398550409 +0000 UTC m=+41.117513529 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert") pod "ingress-canary-lftth" (UID: "95c09ce4-ed3a-4de9-baef-5b4e1fed5652") : secret "canary-serving-cert" not found
May 25 13:06:10.398664 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:10.398628    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:10.398751 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:10.398710    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 13:06:10.398751 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:10.398733    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-845c4b7766-sckdj: secret "image-registry-tls" not found
May 25 13:06:10.398812 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:10.398770    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls podName:05ac39e7-0786-4cd3-9b66-05091c2c0c1a nodeName:}" failed. No retries permitted until 2026-05-25 13:06:14.398758576 +0000 UTC m=+41.117721710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls") pod "image-registry-845c4b7766-sckdj" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a") : secret "image-registry-tls" not found
May 25 13:06:11.051924 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:11.051891    2559 generic.go:358] "Generic (PLEG): container finished" podID="913e5f01-6ff3-4bd5-965e-809823df7d6d" containerID="a67a472febe21dcb4f8f01e36f7d7cd6b7e6764ddc32ae5282ba39a98891d376" exitCode=0
May 25 13:06:11.052349 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:11.051969    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lvfw" event={"ID":"913e5f01-6ff3-4bd5-965e-809823df7d6d","Type":"ContainerDied","Data":"a67a472febe21dcb4f8f01e36f7d7cd6b7e6764ddc32ae5282ba39a98891d376"}
May 25 13:06:12.056648 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:12.056611    2559 generic.go:358] "Generic (PLEG): container finished" podID="913e5f01-6ff3-4bd5-965e-809823df7d6d" containerID="8e7b3a9294e7ee3e6b2af785a98423c30926fe23f26fa0a1fadf5b2c82572a48" exitCode=0
May 25 13:06:12.056648 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:12.056658    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lvfw" event={"ID":"913e5f01-6ff3-4bd5-965e-809823df7d6d","Type":"ContainerDied","Data":"8e7b3a9294e7ee3e6b2af785a98423c30926fe23f26fa0a1fadf5b2c82572a48"}
May 25 13:06:13.060947 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:13.060915    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9lvfw" event={"ID":"913e5f01-6ff3-4bd5-965e-809823df7d6d","Type":"ContainerStarted","Data":"702fbc857adbddae1e52860f1a1186ffc049df13517f24a9386f46609f77ff3f"}
May 25 13:06:13.083443 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:13.083392    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9lvfw" podStartSLOduration=6.626054647 podStartE2EDuration="40.083377518s" podCreationTimestamp="2026-05-25 13:05:33 +0000 UTC" firstStartedPulling="2026-05-25 13:05:36.410423258 +0000 UTC m=+3.129386377" lastFinishedPulling="2026-05-25 13:06:09.86774613 +0000 UTC m=+36.586709248" observedRunningTime="2026-05-25 13:06:13.081977898 +0000 UTC m=+39.800941050" watchObservedRunningTime="2026-05-25 13:06:13.083377518 +0000 UTC m=+39.802340682"
May 25 13:06:14.435183 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:14.435144    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:14.435183 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:14.435193    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert\") pod \"ingress-canary-lftth\" (UID: \"95c09ce4-ed3a-4de9-baef-5b4e1fed5652\") " pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:14.435628 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:14.435308    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 13:06:14.435628 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:14.435334    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-845c4b7766-sckdj: secret "image-registry-tls" not found
May 25 13:06:14.435628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:14.435343    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:14.435628 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:14.435396    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls podName:05ac39e7-0786-4cd3-9b66-05091c2c0c1a nodeName:}" failed. No retries permitted until 2026-05-25 13:06:22.435378086 +0000 UTC m=+49.154341204 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls") pod "image-registry-845c4b7766-sckdj" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a") : secret "image-registry-tls" not found
May 25 13:06:14.435628 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:14.435417    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 13:06:14.435628 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:14.435447    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 13:06:14.435628 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:14.435456    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert podName:95c09ce4-ed3a-4de9-baef-5b4e1fed5652 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:22.435445377 +0000 UTC m=+49.154408495 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert") pod "ingress-canary-lftth" (UID: "95c09ce4-ed3a-4de9-baef-5b4e1fed5652") : secret "canary-serving-cert" not found
May 25 13:06:14.435628 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:14.435483    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls podName:7844394a-a353-4e6f-9c29-75e111adcdb9 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:22.435473269 +0000 UTC m=+49.154436396 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls") pod "dns-default-f6lwr" (UID: "7844394a-a353-4e6f-9c29-75e111adcdb9") : secret "dns-default-metrics-tls" not found
May 25 13:06:17.390729 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.390682    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-zcpd6"]
May 25 13:06:17.411504 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.411477    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-zcpd6"]
May 25 13:06:17.411656 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.411595    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-zcpd6"
May 25 13:06:17.413865 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.413840    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
May 25 13:06:17.413979 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.413892    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
May 25 13:06:17.414599 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.414580    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-q6xnl\""
May 25 13:06:17.555811 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.555764    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q76j9\" (UniqueName: \"kubernetes.io/projected/9ff1dfe7-ee99-4531-a9fc-ace686d55b62-kube-api-access-q76j9\") pod \"migrator-b985f5d89-zcpd6\" (UID: \"9ff1dfe7-ee99-4531-a9fc-ace686d55b62\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-zcpd6"
May 25 13:06:17.656792 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.656673    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q76j9\" (UniqueName: \"kubernetes.io/projected/9ff1dfe7-ee99-4531-a9fc-ace686d55b62-kube-api-access-q76j9\") pod \"migrator-b985f5d89-zcpd6\" (UID: \"9ff1dfe7-ee99-4531-a9fc-ace686d55b62\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-zcpd6"
May 25 13:06:17.664995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.664969    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q76j9\" (UniqueName: \"kubernetes.io/projected/9ff1dfe7-ee99-4531-a9fc-ace686d55b62-kube-api-access-q76j9\") pod \"migrator-b985f5d89-zcpd6\" (UID: \"9ff1dfe7-ee99-4531-a9fc-ace686d55b62\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-zcpd6"
May 25 13:06:17.720218 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.720177    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-zcpd6"
May 25 13:06:17.791056 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.791023    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"]
May 25 13:06:17.815625 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.813912    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"]
May 25 13:06:17.815625 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.814060    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"
May 25 13:06:17.818241 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.816991    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 25 13:06:17.818241 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.817251    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 25 13:06:17.818241 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.817628    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-kbhw5\""
May 25 13:06:17.890065 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.890032    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-zcpd6"]
May 25 13:06:17.893295 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:17.893259    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff1dfe7_ee99_4531_a9fc_ace686d55b62.slice/crio-7e4e615afdeb63ed33f0616c38595b7d93d2f66e486f44b7cab5c4552f9c65f3 WatchSource:0}: Error finding container 7e4e615afdeb63ed33f0616c38595b7d93d2f66e486f44b7cab5c4552f9c65f3: Status 404 returned error can't find the container with id 7e4e615afdeb63ed33f0616c38595b7d93d2f66e486f44b7cab5c4552f9c65f3
May 25 13:06:17.959042 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.958952    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/97b70200-e3c7-46a5-b830-91346bf7cb88-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-kb7wb\" (UID: \"97b70200-e3c7-46a5-b830-91346bf7cb88\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"
May 25 13:06:17.959042 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:17.958993    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-kb7wb\" (UID: \"97b70200-e3c7-46a5-b830-91346bf7cb88\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"
May 25 13:06:18.060221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:18.060189    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/97b70200-e3c7-46a5-b830-91346bf7cb88-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-kb7wb\" (UID: \"97b70200-e3c7-46a5-b830-91346bf7cb88\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"
May 25 13:06:18.060221 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:18.060231    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-kb7wb\" (UID: \"97b70200-e3c7-46a5-b830-91346bf7cb88\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"
May 25 13:06:18.060416 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:18.060362    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 25 13:06:18.060451 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:18.060433    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert podName:97b70200-e3c7-46a5-b830-91346bf7cb88 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:18.560414053 +0000 UTC m=+45.279377174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-kb7wb" (UID: "97b70200-e3c7-46a5-b830-91346bf7cb88") : secret "networking-console-plugin-cert" not found
May 25 13:06:18.060936 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:18.060920    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/97b70200-e3c7-46a5-b830-91346bf7cb88-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-kb7wb\" (UID: \"97b70200-e3c7-46a5-b830-91346bf7cb88\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"
May 25 13:06:18.070375 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:18.070343    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-zcpd6" event={"ID":"9ff1dfe7-ee99-4531-a9fc-ace686d55b62","Type":"ContainerStarted","Data":"7e4e615afdeb63ed33f0616c38595b7d93d2f66e486f44b7cab5c4552f9c65f3"}
May 25 13:06:18.564944 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:18.564898    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-kb7wb\" (UID: \"97b70200-e3c7-46a5-b830-91346bf7cb88\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"
May 25 13:06:18.565511 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:18.565076    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 25 13:06:18.565511 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:18.565146    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert podName:97b70200-e3c7-46a5-b830-91346bf7cb88 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:19.565127394 +0000 UTC m=+46.284090511 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-kb7wb" (UID: "97b70200-e3c7-46a5-b830-91346bf7cb88") : secret "networking-console-plugin-cert" not found
May 25 13:06:19.122370 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.122333    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-nwqcj"]
May 25 13:06:19.142556 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.142526    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-nwqcj"]
May 25 13:06:19.142742 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.142676    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-nwqcj"
May 25 13:06:19.145890 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.145861    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zzr5c\""
May 25 13:06:19.146005 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.145914    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
May 25 13:06:19.146428 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.146406    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
May 25 13:06:19.146538 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.146510    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
May 25 13:06:19.146603 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.146555    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
May 25 13:06:19.270921 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.270883    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5e92e02d-f23f-4739-9ef5-8f7cde37aa1f-signing-key\") pod \"service-ca-67d667c7d-nwqcj\" (UID: \"5e92e02d-f23f-4739-9ef5-8f7cde37aa1f\") " pod="openshift-service-ca/service-ca-67d667c7d-nwqcj"
May 25 13:06:19.271107 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.270949    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5e92e02d-f23f-4739-9ef5-8f7cde37aa1f-signing-cabundle\") pod \"service-ca-67d667c7d-nwqcj\" (UID: \"5e92e02d-f23f-4739-9ef5-8f7cde37aa1f\") " pod="openshift-service-ca/service-ca-67d667c7d-nwqcj"
May 25 13:06:19.271107 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.270978    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvvzr\" (UniqueName: \"kubernetes.io/projected/5e92e02d-f23f-4739-9ef5-8f7cde37aa1f-kube-api-access-rvvzr\") pod \"service-ca-67d667c7d-nwqcj\" (UID: \"5e92e02d-f23f-4739-9ef5-8f7cde37aa1f\") " pod="openshift-service-ca/service-ca-67d667c7d-nwqcj"
May 25 13:06:19.372273 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.372237    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5e92e02d-f23f-4739-9ef5-8f7cde37aa1f-signing-key\") pod \"service-ca-67d667c7d-nwqcj\" (UID: \"5e92e02d-f23f-4739-9ef5-8f7cde37aa1f\") " pod="openshift-service-ca/service-ca-67d667c7d-nwqcj"
May 25 13:06:19.372493 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.372308    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5e92e02d-f23f-4739-9ef5-8f7cde37aa1f-signing-cabundle\") pod \"service-ca-67d667c7d-nwqcj\" (UID: \"5e92e02d-f23f-4739-9ef5-8f7cde37aa1f\") " pod="openshift-service-ca/service-ca-67d667c7d-nwqcj"
May 25 13:06:19.372493 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.372326    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rvvzr\" (UniqueName: \"kubernetes.io/projected/5e92e02d-f23f-4739-9ef5-8f7cde37aa1f-kube-api-access-rvvzr\") pod \"service-ca-67d667c7d-nwqcj\" (UID: \"5e92e02d-f23f-4739-9ef5-8f7cde37aa1f\") " pod="openshift-service-ca/service-ca-67d667c7d-nwqcj"
May 25 13:06:19.373024 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.372998    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5e92e02d-f23f-4739-9ef5-8f7cde37aa1f-signing-cabundle\") pod \"service-ca-67d667c7d-nwqcj\" (UID: \"5e92e02d-f23f-4739-9ef5-8f7cde37aa1f\") " pod="openshift-service-ca/service-ca-67d667c7d-nwqcj"
May 25 13:06:19.374588 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.374562    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5e92e02d-f23f-4739-9ef5-8f7cde37aa1f-signing-key\") pod \"service-ca-67d667c7d-nwqcj\" (UID: \"5e92e02d-f23f-4739-9ef5-8f7cde37aa1f\") " pod="openshift-service-ca/service-ca-67d667c7d-nwqcj"
May 25 13:06:19.382349 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.382328    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvvzr\" (UniqueName: \"kubernetes.io/projected/5e92e02d-f23f-4739-9ef5-8f7cde37aa1f-kube-api-access-rvvzr\") pod \"service-ca-67d667c7d-nwqcj\" (UID: \"5e92e02d-f23f-4739-9ef5-8f7cde37aa1f\") " pod="openshift-service-ca/service-ca-67d667c7d-nwqcj"
May 25 13:06:19.453359 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.453320    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-nwqcj"
May 25 13:06:19.575148 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.575037    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-kb7wb\" (UID: \"97b70200-e3c7-46a5-b830-91346bf7cb88\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"
May 25 13:06:19.575528 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:19.575224    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 25 13:06:19.575528 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:19.575290    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert podName:97b70200-e3c7-46a5-b830-91346bf7cb88 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:21.575270747 +0000 UTC m=+48.294233880 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-kb7wb" (UID: "97b70200-e3c7-46a5-b830-91346bf7cb88") : secret "networking-console-plugin-cert" not found
May 25 13:06:19.639902 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.639874    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-482g6_9ea8b5d3-ab26-4393-b5f9-747d95405510/dns-node-resolver/0.log"
May 25 13:06:19.658218 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:19.658194    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-nwqcj"]
May 25 13:06:19.662040 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:19.662016    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e92e02d_f23f_4739_9ef5_8f7cde37aa1f.slice/crio-bf7a80752fccddf160e5052bc8f23e6a93fcd8e285489055ae0fba3dc52d0be3 WatchSource:0}: Error finding container bf7a80752fccddf160e5052bc8f23e6a93fcd8e285489055ae0fba3dc52d0be3: Status 404 returned error can't find the container with id bf7a80752fccddf160e5052bc8f23e6a93fcd8e285489055ae0fba3dc52d0be3
May 25 13:06:20.076868 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:20.076787    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-zcpd6" event={"ID":"9ff1dfe7-ee99-4531-a9fc-ace686d55b62","Type":"ContainerStarted","Data":"c6c75ffd9d640c4e1d6096b08b20a6f825c3dfb97ea04e66719f7fab47c27e02"}
May 25 13:06:20.076868 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:20.076842    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-zcpd6" event={"ID":"9ff1dfe7-ee99-4531-a9fc-ace686d55b62","Type":"ContainerStarted","Data":"196be9d65ef49c308fc8c84b2a27a266c20e1c4c90bf22f581bf784297284c03"}
May 25 13:06:20.078239 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:20.078205    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-nwqcj" event={"ID":"5e92e02d-f23f-4739-9ef5-8f7cde37aa1f","Type":"ContainerStarted","Data":"bf7a80752fccddf160e5052bc8f23e6a93fcd8e285489055ae0fba3dc52d0be3"}
May 25 13:06:20.094126 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:20.094051    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-zcpd6" podStartSLOduration=1.464723605 podStartE2EDuration="3.094031996s" podCreationTimestamp="2026-05-25 13:06:17 +0000 UTC" firstStartedPulling="2026-05-25 13:06:17.895754467 +0000 UTC m=+44.614717586" lastFinishedPulling="2026-05-25 13:06:19.525062849 +0000 UTC m=+46.244025977" observedRunningTime="2026-05-25 13:06:20.093836522 +0000 UTC m=+46.812799665" watchObservedRunningTime="2026-05-25 13:06:20.094031996 +0000 UTC m=+46.812995137"
May 25 13:06:20.179671 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:20.179627    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:06:20.182193 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:20.182166    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbb8062f-fbfd-4af1-ae8d-c7fb90612985-original-pull-secret\") pod \"global-pull-secret-syncer-xfbms\" (UID: \"fbb8062f-fbfd-4af1-ae8d-c7fb90612985\") " pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:06:20.375886 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:20.375808    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xfbms"
May 25 13:06:20.531171 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:20.531133    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xfbms"]
May 25 13:06:20.534172 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:20.534143    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbb8062f_fbfd_4af1_ae8d_c7fb90612985.slice/crio-7b1ac269ffac3e4e8407a8377020f9e10a4781ab850121065b08256ea9df4765 WatchSource:0}: Error finding container 7b1ac269ffac3e4e8407a8377020f9e10a4781ab850121065b08256ea9df4765: Status 404 returned error can't find the container with id 7b1ac269ffac3e4e8407a8377020f9e10a4781ab850121065b08256ea9df4765
May 25 13:06:20.640704 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:20.640630    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-95pwd_bc716b46-7c31-42f5-ba6f-45b8c77d29ad/node-ca/0.log"
May 25 13:06:21.082455 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:21.082414    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xfbms" event={"ID":"fbb8062f-fbfd-4af1-ae8d-c7fb90612985","Type":"ContainerStarted","Data":"7b1ac269ffac3e4e8407a8377020f9e10a4781ab850121065b08256ea9df4765"}
May 25 13:06:21.592966 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:21.592927    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-kb7wb\" (UID: \"97b70200-e3c7-46a5-b830-91346bf7cb88\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"
May 25 13:06:21.593136 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:21.593056    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 25 13:06:21.593136 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:21.593120    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert podName:97b70200-e3c7-46a5-b830-91346bf7cb88 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:25.593100407 +0000 UTC m=+52.312063541 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-kb7wb" (UID: "97b70200-e3c7-46a5-b830-91346bf7cb88") : secret "networking-console-plugin-cert" not found
May 25 13:06:21.840943 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:21.840911    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-zcpd6_9ff1dfe7-ee99-4531-a9fc-ace686d55b62/migrator/0.log"
May 25 13:06:22.040327 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:22.040264    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-zcpd6_9ff1dfe7-ee99-4531-a9fc-ace686d55b62/graceful-termination/0.log"
May 25 13:06:22.089810 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:22.089767    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-nwqcj" event={"ID":"5e92e02d-f23f-4739-9ef5-8f7cde37aa1f","Type":"ContainerStarted","Data":"fda88568abe308f09b2641d72aaee373f835ae60243c9d94d866e55d5c60bf32"}
May 25 13:06:22.109768 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:22.109589    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-67d667c7d-nwqcj" podStartSLOduration=1.629232164 podStartE2EDuration="3.109569237s" podCreationTimestamp="2026-05-25 13:06:19 +0000 UTC" firstStartedPulling="2026-05-25 13:06:19.663870262 +0000 UTC m=+46.382833380" lastFinishedPulling="2026-05-25 13:06:21.144207336 +0000 UTC m=+47.863170453" observedRunningTime="2026-05-25 13:06:22.109422083 +0000 UTC m=+48.828385224" watchObservedRunningTime="2026-05-25 13:06:22.109569237 +0000 UTC m=+48.828532378"
May 25 13:06:22.500189 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:22.499491    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:22.500189 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:22.499556    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert\") pod \"ingress-canary-lftth\" (UID: \"95c09ce4-ed3a-4de9-baef-5b4e1fed5652\") " pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:22.500189 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:22.499615    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:22.500189 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:22.499783    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 13:06:22.500189 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:22.499846    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls podName:7844394a-a353-4e6f-9c29-75e111adcdb9 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:38.499826229 +0000 UTC m=+65.218789352 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls") pod "dns-default-f6lwr" (UID: "7844394a-a353-4e6f-9c29-75e111adcdb9") : secret "dns-default-metrics-tls" not found
May 25 13:06:22.500189 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:22.499945    2559 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 13:06:22.500189 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:22.499987    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert podName:95c09ce4-ed3a-4de9-baef-5b4e1fed5652 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:38.499974993 +0000 UTC m=+65.218938114 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert") pod "ingress-canary-lftth" (UID: "95c09ce4-ed3a-4de9-baef-5b4e1fed5652") : secret "canary-serving-cert" not found
May 25 13:06:22.500189 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:22.500040    2559 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 13:06:22.500189 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:22.500051    2559 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-845c4b7766-sckdj: secret "image-registry-tls" not found
May 25 13:06:22.500189 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:22.500083    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls podName:05ac39e7-0786-4cd3-9b66-05091c2c0c1a nodeName:}" failed. No retries permitted until 2026-05-25 13:06:38.500072703 +0000 UTC m=+65.219035834 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls") pod "image-registry-845c4b7766-sckdj" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a") : secret "image-registry-tls" not found
May 25 13:06:25.097708 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:25.097667    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xfbms" event={"ID":"fbb8062f-fbfd-4af1-ae8d-c7fb90612985","Type":"ContainerStarted","Data":"7996566d6f83bb04f50a9a2ab8bd4698eb03b1e0fc9444f90b46d204ff5e1661"}
May 25 13:06:25.114195 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:25.114135    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xfbms" podStartSLOduration=32.995340884 podStartE2EDuration="37.114113315s" podCreationTimestamp="2026-05-25 13:05:48 +0000 UTC" firstStartedPulling="2026-05-25 13:06:20.535993892 +0000 UTC m=+47.254957023" lastFinishedPulling="2026-05-25 13:06:24.654766337 +0000 UTC m=+51.373729454" observedRunningTime="2026-05-25 13:06:25.112965747 +0000 UTC m=+51.831928888" watchObservedRunningTime="2026-05-25 13:06:25.114113315 +0000 UTC m=+51.833076455"
May 25 13:06:25.628423 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:25.628384    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-kb7wb\" (UID: \"97b70200-e3c7-46a5-b830-91346bf7cb88\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"
May 25 13:06:25.628619 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:25.628559    2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 25 13:06:25.628664 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:06:25.628638    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert podName:97b70200-e3c7-46a5-b830-91346bf7cb88 nodeName:}" failed. No retries permitted until 2026-05-25 13:06:33.628617906 +0000 UTC m=+60.347581036 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-kb7wb" (UID: "97b70200-e3c7-46a5-b830-91346bf7cb88") : secret "networking-console-plugin-cert" not found
May 25 13:06:33.701113 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:33.701069    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-kb7wb\" (UID: \"97b70200-e3c7-46a5-b830-91346bf7cb88\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"
May 25 13:06:33.703451 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:33.703427    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/97b70200-e3c7-46a5-b830-91346bf7cb88-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-kb7wb\" (UID: \"97b70200-e3c7-46a5-b830-91346bf7cb88\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"
May 25 13:06:33.742465 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:33.742435    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-kbhw5\""
May 25 13:06:33.752551 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:33.752508    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"
May 25 13:06:33.878596 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:33.878567    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb"]
May 25 13:06:33.882396 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:33.882366    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b70200_e3c7_46a5_b830_91346bf7cb88.slice/crio-cc958a31a0c3d7eab0bea1768a4c7324f5f7ec3391193f9fb713e6ae3d4c090c WatchSource:0}: Error finding container cc958a31a0c3d7eab0bea1768a4c7324f5f7ec3391193f9fb713e6ae3d4c090c: Status 404 returned error can't find the container with id cc958a31a0c3d7eab0bea1768a4c7324f5f7ec3391193f9fb713e6ae3d4c090c
May 25 13:06:34.116256 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:34.116215    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb" event={"ID":"97b70200-e3c7-46a5-b830-91346bf7cb88","Type":"ContainerStarted","Data":"cc958a31a0c3d7eab0bea1768a4c7324f5f7ec3391193f9fb713e6ae3d4c090c"}
May 25 13:06:34.830993 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:34.830971    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x4vqs"
May 25 13:06:35.119481 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:35.119401    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb" event={"ID":"97b70200-e3c7-46a5-b830-91346bf7cb88","Type":"ContainerStarted","Data":"5ad9f58907c297d9ff3cdc6d08b7aab8ca3779f7e076a903082277aa3b5ef65a"}
May 25 13:06:35.135399 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:35.135349    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-kb7wb" podStartSLOduration=17.221721554 podStartE2EDuration="18.135332129s" podCreationTimestamp="2026-05-25 13:06:17 +0000 UTC" firstStartedPulling="2026-05-25 13:06:33.884747179 +0000 UTC m=+60.603710295" lastFinishedPulling="2026-05-25 13:06:34.798357749 +0000 UTC m=+61.517320870" observedRunningTime="2026-05-25 13:06:35.134463923 +0000 UTC m=+61.853427063" watchObservedRunningTime="2026-05-25 13:06:35.135332129 +0000 UTC m=+61.854295314"
May 25 13:06:38.544346 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.544304    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:38.544828 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.544356    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert\") pod \"ingress-canary-lftth\" (UID: \"95c09ce4-ed3a-4de9-baef-5b4e1fed5652\") " pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:38.544828 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.544394    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:38.546636 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.546613    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7844394a-a353-4e6f-9c29-75e111adcdb9-metrics-tls\") pod \"dns-default-f6lwr\" (UID: \"7844394a-a353-4e6f-9c29-75e111adcdb9\") " pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:38.546751 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.546700    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls\") pod \"image-registry-845c4b7766-sckdj\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") " pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:38.546814 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.546752    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95c09ce4-ed3a-4de9-baef-5b4e1fed5652-cert\") pod \"ingress-canary-lftth\" (UID: \"95c09ce4-ed3a-4de9-baef-5b4e1fed5652\") " pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:38.769871 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.769836    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lntzn\""
May 25 13:06:38.777999 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.777974    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:38.786270 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.786245    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vbzlk\""
May 25 13:06:38.794125 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.794095    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lftth"
May 25 13:06:38.802658 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.802636    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5d4pm\""
May 25 13:06:38.810712 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.810687    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:38.934312 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.934281    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-845c4b7766-sckdj"]
May 25 13:06:38.939199 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:38.939166    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ac39e7_0786_4cd3_9b66_05091c2c0c1a.slice/crio-d1f1883a8aa62d125548bf6798b59cd06911816ad2f4f1ffab4500018b69762b WatchSource:0}: Error finding container d1f1883a8aa62d125548bf6798b59cd06911816ad2f4f1ffab4500018b69762b: Status 404 returned error can't find the container with id d1f1883a8aa62d125548bf6798b59cd06911816ad2f4f1ffab4500018b69762b
May 25 13:06:38.953179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.953150    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lftth"]
May 25 13:06:38.956298 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:38.956272    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95c09ce4_ed3a_4de9_baef_5b4e1fed5652.slice/crio-56cd3f36d2b218b7abc942fa8a89d2280daaaa56681fd78d535544868ec9b5ed WatchSource:0}: Error finding container 56cd3f36d2b218b7abc942fa8a89d2280daaaa56681fd78d535544868ec9b5ed: Status 404 returned error can't find the container with id 56cd3f36d2b218b7abc942fa8a89d2280daaaa56681fd78d535544868ec9b5ed
May 25 13:06:38.977512 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:38.977484    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f6lwr"]
May 25 13:06:38.980598 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:38.980572    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7844394a_a353_4e6f_9c29_75e111adcdb9.slice/crio-a26924521edc0162c1af8921f1ce231404778b48c32d9a6228fdc37d2e810a65 WatchSource:0}: Error finding container a26924521edc0162c1af8921f1ce231404778b48c32d9a6228fdc37d2e810a65: Status 404 returned error can't find the container with id a26924521edc0162c1af8921f1ce231404778b48c32d9a6228fdc37d2e810a65
May 25 13:06:39.129837 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.129701    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-845c4b7766-sckdj" event={"ID":"05ac39e7-0786-4cd3-9b66-05091c2c0c1a","Type":"ContainerStarted","Data":"3c21ba7adf47351843fbe9784e9e11577cabd6c9323da1c33b851b89ebc0e22a"}
May 25 13:06:39.129837 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.129773    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-845c4b7766-sckdj" event={"ID":"05ac39e7-0786-4cd3-9b66-05091c2c0c1a","Type":"ContainerStarted","Data":"d1f1883a8aa62d125548bf6798b59cd06911816ad2f4f1ffab4500018b69762b"}
May 25 13:06:39.129837 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.129825    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:06:39.130803 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.130770    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f6lwr" event={"ID":"7844394a-a353-4e6f-9c29-75e111adcdb9","Type":"ContainerStarted","Data":"a26924521edc0162c1af8921f1ce231404778b48c32d9a6228fdc37d2e810a65"}
May 25 13:06:39.131904 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.131882    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lftth" event={"ID":"95c09ce4-ed3a-4de9-baef-5b4e1fed5652","Type":"ContainerStarted","Data":"56cd3f36d2b218b7abc942fa8a89d2280daaaa56681fd78d535544868ec9b5ed"}
May 25 13:06:39.552077 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.552037    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs\") pod \"network-metrics-daemon-tx5hr\" (UID: \"fde8fc4f-07bb-442f-aa5e-6e63efece856\") " pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:06:39.555476 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.555453    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 25 13:06:39.564761 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.564742    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fde8fc4f-07bb-442f-aa5e-6e63efece856-metrics-certs\") pod \"network-metrics-daemon-tx5hr\" (UID: \"fde8fc4f-07bb-442f-aa5e-6e63efece856\") " pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:06:39.653115 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.653080    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k99qx\" (UniqueName: \"kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx\") pod \"network-check-target-z7xmf\" (UID: \"d4c3716c-3120-42b7-a927-bca86e4b5383\") " pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:06:39.655644 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.655621    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 25 13:06:39.670568 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.670538    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 25 13:06:39.676920 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.676862    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k99qx\" (UniqueName: \"kubernetes.io/projected/d4c3716c-3120-42b7-a927-bca86e4b5383-kube-api-access-k99qx\") pod \"network-check-target-z7xmf\" (UID: \"d4c3716c-3120-42b7-a927-bca86e4b5383\") " pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:06:39.684948 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.684925    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gcddr\""
May 25 13:06:39.686021 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.686002    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-clhmx\""
May 25 13:06:39.688510 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.688492    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:06:39.693745 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.693688    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tx5hr"
May 25 13:06:39.847673 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.847136    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-845c4b7766-sckdj" podStartSLOduration=48.847116832 podStartE2EDuration="48.847116832s" podCreationTimestamp="2026-05-25 13:05:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:06:39.160051611 +0000 UTC m=+65.879014747" watchObservedRunningTime="2026-05-25 13:06:39.847116832 +0000 UTC m=+66.566079972"
May 25 13:06:39.847673 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.847627    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z7xmf"]
May 25 13:06:39.851072 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:39.851042    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4c3716c_3120_42b7_a927_bca86e4b5383.slice/crio-e1699dcb43e9a1ffeb2fab0e2272c08ba78bacd0ef43ffa3dd8264812926b021 WatchSource:0}: Error finding container e1699dcb43e9a1ffeb2fab0e2272c08ba78bacd0ef43ffa3dd8264812926b021: Status 404 returned error can't find the container with id e1699dcb43e9a1ffeb2fab0e2272c08ba78bacd0ef43ffa3dd8264812926b021
May 25 13:06:39.869107 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:39.869085    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tx5hr"]
May 25 13:06:39.871428 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:39.871387    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde8fc4f_07bb_442f_aa5e_6e63efece856.slice/crio-8511e5a8b5859237cc573f37e401d5490276c35ff2bc5202b100d3206b3f1fc3 WatchSource:0}: Error finding container 8511e5a8b5859237cc573f37e401d5490276c35ff2bc5202b100d3206b3f1fc3: Status 404 returned error can't find the container with id 8511e5a8b5859237cc573f37e401d5490276c35ff2bc5202b100d3206b3f1fc3
May 25 13:06:40.136764 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:40.136642    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z7xmf" event={"ID":"d4c3716c-3120-42b7-a927-bca86e4b5383","Type":"ContainerStarted","Data":"e1699dcb43e9a1ffeb2fab0e2272c08ba78bacd0ef43ffa3dd8264812926b021"}
May 25 13:06:40.138016 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:40.137970    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tx5hr" event={"ID":"fde8fc4f-07bb-442f-aa5e-6e63efece856","Type":"ContainerStarted","Data":"8511e5a8b5859237cc573f37e401d5490276c35ff2bc5202b100d3206b3f1fc3"}
May 25 13:06:41.079352 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.079307    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n7bht"]
May 25 13:06:41.095834 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.095760    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"]
May 25 13:06:41.096001 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.095875    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.099910 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.099830    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 25 13:06:41.099910 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.099879    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wgqp9\""
May 25 13:06:41.100409 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.100364    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 25 13:06:41.101546 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.101090    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 25 13:06:41.101546 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.101384    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 25 13:06:41.109765 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.109735    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9"]
May 25 13:06:41.110075 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.109969    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"
May 25 13:06:41.115395 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.115372    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 25 13:06:41.115517 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.115373    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 25 13:06:41.115517 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.115428    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
May 25 13:06:41.115517 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.115446    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 25 13:06:41.127999 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.127897    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9"]
May 25 13:06:41.127999 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.127929    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n7bht"]
May 25 13:06:41.127999 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.127942    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"]
May 25 13:06:41.127999 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.127959    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-845c4b7766-sckdj"]
May 25 13:06:41.128294 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.128064    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9"
May 25 13:06:41.130886 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.130861    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
May 25 13:06:41.131007 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.130861    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-kshqc\""
May 25 13:06:41.164863 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.164820    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffdm2\" (UniqueName: \"kubernetes.io/projected/b57c7dbd-b155-4c76-bde0-22f1b87cb7de-kube-api-access-ffdm2\") pod \"managed-serviceaccount-addon-agent-bfc749cb4-98jb9\" (UID: \"b57c7dbd-b155-4c76-bde0-22f1b87cb7de\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9"
May 25 13:06:41.165035 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.164877    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/aaa33289-3ee5-4ad2-99a7-e2b158714252-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d8b8486dc-7lcj7\" (UID: \"aaa33289-3ee5-4ad2-99a7-e2b158714252\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"
May 25 13:06:41.165035 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.164907    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b57c7dbd-b155-4c76-bde0-22f1b87cb7de-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-bfc749cb4-98jb9\" (UID: \"b57c7dbd-b155-4c76-bde0-22f1b87cb7de\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9"
May 25 13:06:41.165035 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.164946    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/989dfcac-2684-4ca2-929d-947ffe34fba1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.165035 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.164975    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aaa33289-3ee5-4ad2-99a7-e2b158714252-tmp\") pod \"klusterlet-addon-workmgr-7d8b8486dc-7lcj7\" (UID: \"aaa33289-3ee5-4ad2-99a7-e2b158714252\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"
May 25 13:06:41.165035 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.165005    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l49bm\" (UniqueName: \"kubernetes.io/projected/aaa33289-3ee5-4ad2-99a7-e2b158714252-kube-api-access-l49bm\") pod \"klusterlet-addon-workmgr-7d8b8486dc-7lcj7\" (UID: \"aaa33289-3ee5-4ad2-99a7-e2b158714252\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"
May 25 13:06:41.165035 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.165030    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/989dfcac-2684-4ca2-929d-947ffe34fba1-data-volume\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.165328 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.165057    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/989dfcac-2684-4ca2-929d-947ffe34fba1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.165328 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.165094    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zkq7\" (UniqueName: \"kubernetes.io/projected/989dfcac-2684-4ca2-929d-947ffe34fba1-kube-api-access-6zkq7\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.165328 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.165131    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/989dfcac-2684-4ca2-929d-947ffe34fba1-crio-socket\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.197498 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.197466    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-57bd96b888-rfml4"]
May 25 13:06:41.220100 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.220067    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-rfml4"
May 25 13:06:41.222551 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.222489    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9564m\""
May 25 13:06:41.222703 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.222597    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 25 13:06:41.222809 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.222754    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 25 13:06:41.233902 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.232465    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-rfml4"]
May 25 13:06:41.266032 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.265998    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/989dfcac-2684-4ca2-929d-947ffe34fba1-crio-socket\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.266219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.266056    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ffdm2\" (UniqueName: \"kubernetes.io/projected/b57c7dbd-b155-4c76-bde0-22f1b87cb7de-kube-api-access-ffdm2\") pod \"managed-serviceaccount-addon-agent-bfc749cb4-98jb9\" (UID: \"b57c7dbd-b155-4c76-bde0-22f1b87cb7de\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9"
May 25 13:06:41.266219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.266095    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/aaa33289-3ee5-4ad2-99a7-e2b158714252-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d8b8486dc-7lcj7\" (UID: \"aaa33289-3ee5-4ad2-99a7-e2b158714252\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"
May 25 13:06:41.266219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.266124    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b57c7dbd-b155-4c76-bde0-22f1b87cb7de-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-bfc749cb4-98jb9\" (UID: \"b57c7dbd-b155-4c76-bde0-22f1b87cb7de\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9"
May 25 13:06:41.266219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.266157    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/989dfcac-2684-4ca2-929d-947ffe34fba1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.266219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.266183    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aaa33289-3ee5-4ad2-99a7-e2b158714252-tmp\") pod \"klusterlet-addon-workmgr-7d8b8486dc-7lcj7\" (UID: \"aaa33289-3ee5-4ad2-99a7-e2b158714252\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"
May 25 13:06:41.266219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.266216    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8lrf\" (UniqueName: \"kubernetes.io/projected/a80cd8c7-074e-47ef-8de3-f2921430ad60-kube-api-access-s8lrf\") pod \"downloads-57bd96b888-rfml4\" (UID: \"a80cd8c7-074e-47ef-8de3-f2921430ad60\") " pod="openshift-console/downloads-57bd96b888-rfml4"
May 25 13:06:41.266515 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.266245    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l49bm\" (UniqueName: \"kubernetes.io/projected/aaa33289-3ee5-4ad2-99a7-e2b158714252-kube-api-access-l49bm\") pod \"klusterlet-addon-workmgr-7d8b8486dc-7lcj7\" (UID: \"aaa33289-3ee5-4ad2-99a7-e2b158714252\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"
May 25 13:06:41.266515 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.266259    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/989dfcac-2684-4ca2-929d-947ffe34fba1-crio-socket\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.266515 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.266271    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/989dfcac-2684-4ca2-929d-947ffe34fba1-data-volume\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.266515 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.266334    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/989dfcac-2684-4ca2-929d-947ffe34fba1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.266515 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.266369    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6zkq7\" (UniqueName: \"kubernetes.io/projected/989dfcac-2684-4ca2-929d-947ffe34fba1-kube-api-access-6zkq7\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.266796 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.266588    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/989dfcac-2684-4ca2-929d-947ffe34fba1-data-volume\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.266940 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.266921    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aaa33289-3ee5-4ad2-99a7-e2b158714252-tmp\") pod \"klusterlet-addon-workmgr-7d8b8486dc-7lcj7\" (UID: \"aaa33289-3ee5-4ad2-99a7-e2b158714252\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"
May 25 13:06:41.267641 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.267556    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/989dfcac-2684-4ca2-929d-947ffe34fba1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.270023 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.269994    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/989dfcac-2684-4ca2-929d-947ffe34fba1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.270140 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.270019    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/aaa33289-3ee5-4ad2-99a7-e2b158714252-klusterlet-config\") pod \"klusterlet-addon-workmgr-7d8b8486dc-7lcj7\" (UID: \"aaa33289-3ee5-4ad2-99a7-e2b158714252\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"
May 25 13:06:41.270140 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.270101    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b57c7dbd-b155-4c76-bde0-22f1b87cb7de-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-bfc749cb4-98jb9\" (UID: \"b57c7dbd-b155-4c76-bde0-22f1b87cb7de\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9"
May 25 13:06:41.294950 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.294893    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l49bm\" (UniqueName: \"kubernetes.io/projected/aaa33289-3ee5-4ad2-99a7-e2b158714252-kube-api-access-l49bm\") pod \"klusterlet-addon-workmgr-7d8b8486dc-7lcj7\" (UID: \"aaa33289-3ee5-4ad2-99a7-e2b158714252\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"
May 25 13:06:41.295648 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.295619    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zkq7\" (UniqueName: \"kubernetes.io/projected/989dfcac-2684-4ca2-929d-947ffe34fba1-kube-api-access-6zkq7\") pod \"insights-runtime-extractor-n7bht\" (UID: \"989dfcac-2684-4ca2-929d-947ffe34fba1\") " pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.297010 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.296986    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffdm2\" (UniqueName: \"kubernetes.io/projected/b57c7dbd-b155-4c76-bde0-22f1b87cb7de-kube-api-access-ffdm2\") pod \"managed-serviceaccount-addon-agent-bfc749cb4-98jb9\" (UID: \"b57c7dbd-b155-4c76-bde0-22f1b87cb7de\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9"
May 25 13:06:41.299807 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.299786    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bbd6764c5-zll4g"]
May 25 13:06:41.332144 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.332070    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bbd6764c5-zll4g"]
May 25 13:06:41.332275 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.332203    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.367544 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.367509    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-766ls\" (UniqueName: \"kubernetes.io/projected/b1f9b575-90ca-41e5-b2c0-c90137e06eac-kube-api-access-766ls\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.367699 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.367566    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1f9b575-90ca-41e5-b2c0-c90137e06eac-ca-trust-extracted\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.367699 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.367595    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1f9b575-90ca-41e5-b2c0-c90137e06eac-registry-tls\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.367699 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.367624    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1f9b575-90ca-41e5-b2c0-c90137e06eac-registry-certificates\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.367699 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.367653    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1f9b575-90ca-41e5-b2c0-c90137e06eac-installation-pull-secrets\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.367969 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.367736    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s8lrf\" (UniqueName: \"kubernetes.io/projected/a80cd8c7-074e-47ef-8de3-f2921430ad60-kube-api-access-s8lrf\") pod \"downloads-57bd96b888-rfml4\" (UID: \"a80cd8c7-074e-47ef-8de3-f2921430ad60\") " pod="openshift-console/downloads-57bd96b888-rfml4"
May 25 13:06:41.367969 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.367782    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1f9b575-90ca-41e5-b2c0-c90137e06eac-image-registry-private-configuration\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.367969 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.367812    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1f9b575-90ca-41e5-b2c0-c90137e06eac-bound-sa-token\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.367969 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.367855    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1f9b575-90ca-41e5-b2c0-c90137e06eac-trusted-ca\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.381441 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.381404    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8lrf\" (UniqueName: \"kubernetes.io/projected/a80cd8c7-074e-47ef-8de3-f2921430ad60-kube-api-access-s8lrf\") pod \"downloads-57bd96b888-rfml4\" (UID: \"a80cd8c7-074e-47ef-8de3-f2921430ad60\") " pod="openshift-console/downloads-57bd96b888-rfml4"
May 25 13:06:41.408879 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.408840    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n7bht"
May 25 13:06:41.422838 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.422810    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"
May 25 13:06:41.447655 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.447625    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9"
May 25 13:06:41.468484 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.468450    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-766ls\" (UniqueName: \"kubernetes.io/projected/b1f9b575-90ca-41e5-b2c0-c90137e06eac-kube-api-access-766ls\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.468626 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.468493    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1f9b575-90ca-41e5-b2c0-c90137e06eac-ca-trust-extracted\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.468626 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.468513    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1f9b575-90ca-41e5-b2c0-c90137e06eac-registry-tls\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.468626 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.468538    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1f9b575-90ca-41e5-b2c0-c90137e06eac-registry-certificates\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.468626 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.468566    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1f9b575-90ca-41e5-b2c0-c90137e06eac-installation-pull-secrets\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.468626 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.468594    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1f9b575-90ca-41e5-b2c0-c90137e06eac-image-registry-private-configuration\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.468626 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.468619    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1f9b575-90ca-41e5-b2c0-c90137e06eac-bound-sa-token\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.468958 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.468653    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1f9b575-90ca-41e5-b2c0-c90137e06eac-trusted-ca\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.468958 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.468940    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b1f9b575-90ca-41e5-b2c0-c90137e06eac-ca-trust-extracted\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.469643 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.469580    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1f9b575-90ca-41e5-b2c0-c90137e06eac-trusted-ca\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.469874 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.469852    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b1f9b575-90ca-41e5-b2c0-c90137e06eac-registry-certificates\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.471697 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.471675    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b1f9b575-90ca-41e5-b2c0-c90137e06eac-installation-pull-secrets\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.471799 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.471677    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b1f9b575-90ca-41e5-b2c0-c90137e06eac-image-registry-private-configuration\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.471887 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.471866    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b1f9b575-90ca-41e5-b2c0-c90137e06eac-registry-tls\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.479353 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.479332    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cdcffffd8-vj5sd"]
May 25 13:06:41.485657 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.485630    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-766ls\" (UniqueName: \"kubernetes.io/projected/b1f9b575-90ca-41e5-b2c0-c90137e06eac-kube-api-access-766ls\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.492645 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.492623    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b1f9b575-90ca-41e5-b2c0-c90137e06eac-bound-sa-token\") pod \"image-registry-bbd6764c5-zll4g\" (UID: \"b1f9b575-90ca-41e5-b2c0-c90137e06eac\") " pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.498647 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.498628    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.505075 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.505054    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 25 13:06:41.505649 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.505620    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 25 13:06:41.505888 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.505817    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 25 13:06:41.505960 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.505934    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qkg9t\""
May 25 13:06:41.506097 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.506076    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 25 13:06:41.506165 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.506112    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 25 13:06:41.517440 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.517413    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cdcffffd8-vj5sd"]
May 25 13:06:41.532954 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.532930    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-rfml4"
May 25 13:06:41.569582 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.569543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-oauth-serving-cert\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.569582 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.569583    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/958511d0-c8a0-4f44-8698-57ad3b96851c-console-oauth-config\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.569815 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.569646    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/958511d0-c8a0-4f44-8698-57ad3b96851c-console-serving-cert\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.569815 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.569711    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-service-ca\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.569815 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.569786    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-console-config\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.569815 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.569803    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrsd5\" (UniqueName: \"kubernetes.io/projected/958511d0-c8a0-4f44-8698-57ad3b96851c-kube-api-access-xrsd5\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.643548 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.643470    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:41.670665 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.670630    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/958511d0-c8a0-4f44-8698-57ad3b96851c-console-oauth-config\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.670847 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.670680    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/958511d0-c8a0-4f44-8698-57ad3b96851c-console-serving-cert\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.670847 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.670745    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-service-ca\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.670847 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.670783    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-console-config\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.670847 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.670809    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xrsd5\" (UniqueName: \"kubernetes.io/projected/958511d0-c8a0-4f44-8698-57ad3b96851c-kube-api-access-xrsd5\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.671047 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.670871    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-oauth-serving-cert\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.671551 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.671525    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-service-ca\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.671649 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.671584    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-oauth-serving-cert\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.671711 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.671659    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-console-config\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.673685 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.673662    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/958511d0-c8a0-4f44-8698-57ad3b96851c-console-oauth-config\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.673798 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.673685    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/958511d0-c8a0-4f44-8698-57ad3b96851c-console-serving-cert\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.683874 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.683848    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrsd5\" (UniqueName: \"kubernetes.io/projected/958511d0-c8a0-4f44-8698-57ad3b96851c-kube-api-access-xrsd5\") pod \"console-5cdcffffd8-vj5sd\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") " pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:41.810478 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:41.810070    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:43.264037 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:43.261902    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n7bht"]
May 25 13:06:43.325437 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:43.325385    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9"]
May 25 13:06:43.325843 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:43.325818    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-rfml4"]
May 25 13:06:43.332215 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:43.332148    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb57c7dbd_b155_4c76_bde0_22f1b87cb7de.slice/crio-a0a7c744f96ac31f8a1f41dacd8e746997a8ef04ab6b8ca4179d6f95284ec0c6 WatchSource:0}: Error finding container a0a7c744f96ac31f8a1f41dacd8e746997a8ef04ab6b8ca4179d6f95284ec0c6: Status 404 returned error can't find the container with id a0a7c744f96ac31f8a1f41dacd8e746997a8ef04ab6b8ca4179d6f95284ec0c6
May 25 13:06:43.340702 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:43.340669    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda80cd8c7_074e_47ef_8de3_f2921430ad60.slice/crio-ba927e480a574ec5d40ec767e97499c5b863f13e3a92fab5efba8d0a35bbbd91 WatchSource:0}: Error finding container ba927e480a574ec5d40ec767e97499c5b863f13e3a92fab5efba8d0a35bbbd91: Status 404 returned error can't find the container with id ba927e480a574ec5d40ec767e97499c5b863f13e3a92fab5efba8d0a35bbbd91
May 25 13:06:43.346743 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:43.346663    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cdcffffd8-vj5sd"]
May 25 13:06:43.352040 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:43.352002    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod958511d0_c8a0_4f44_8698_57ad3b96851c.slice/crio-d1a55e61a380ccd00ea4511b4ceb3619804a8be9a653bad58237d9ae45806f3b WatchSource:0}: Error finding container d1a55e61a380ccd00ea4511b4ceb3619804a8be9a653bad58237d9ae45806f3b: Status 404 returned error can't find the container with id d1a55e61a380ccd00ea4511b4ceb3619804a8be9a653bad58237d9ae45806f3b
May 25 13:06:43.354026 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:43.353825    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"]
May 25 13:06:43.366742 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:43.366685    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa33289_3ee5_4ad2_99a7_e2b158714252.slice/crio-96f964b5ebe92537389ff1b2db4c14e7a086335ea1be682a85ec40e594382d9c WatchSource:0}: Error finding container 96f964b5ebe92537389ff1b2db4c14e7a086335ea1be682a85ec40e594382d9c: Status 404 returned error can't find the container with id 96f964b5ebe92537389ff1b2db4c14e7a086335ea1be682a85ec40e594382d9c
May 25 13:06:43.420048 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:43.420019    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bbd6764c5-zll4g"]
May 25 13:06:43.422291 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:06:43.422262    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1f9b575_90ca_41e5_b2c0_c90137e06eac.slice/crio-cdb3b1d395ab6448157a0e43fac57b816f6ec3b78fc0fcdf5ceaaa1d77fbbddd WatchSource:0}: Error finding container cdb3b1d395ab6448157a0e43fac57b816f6ec3b78fc0fcdf5ceaaa1d77fbbddd: Status 404 returned error can't find the container with id cdb3b1d395ab6448157a0e43fac57b816f6ec3b78fc0fcdf5ceaaa1d77fbbddd
May 25 13:06:44.163561 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.162734    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z7xmf" event={"ID":"d4c3716c-3120-42b7-a927-bca86e4b5383","Type":"ContainerStarted","Data":"3ca033035c9ddd87213dc8daaae1b25fc0ab98b555d7b632d2cbd906a419c053"}
May 25 13:06:44.163561 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.163512    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:06:44.166451 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.166391    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7" event={"ID":"aaa33289-3ee5-4ad2-99a7-e2b158714252","Type":"ContainerStarted","Data":"96f964b5ebe92537389ff1b2db4c14e7a086335ea1be682a85ec40e594382d9c"}
May 25 13:06:44.181804 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.170589    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tx5hr" event={"ID":"fde8fc4f-07bb-442f-aa5e-6e63efece856","Type":"ContainerStarted","Data":"282487b3c45c71d7159dfd7696167d93a85ca0e06fa856b543ffb7f3d30f418a"}
May 25 13:06:44.181804 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.175632    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tx5hr" event={"ID":"fde8fc4f-07bb-442f-aa5e-6e63efece856","Type":"ContainerStarted","Data":"7f9f3d3e1c13cb9ae7f1cf4c62f6e8d37c828e38a36e6b431e84ab12f694e7e0"}
May 25 13:06:44.181804 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.180100    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cdcffffd8-vj5sd" event={"ID":"958511d0-c8a0-4f44-8698-57ad3b96851c","Type":"ContainerStarted","Data":"d1a55e61a380ccd00ea4511b4ceb3619804a8be9a653bad58237d9ae45806f3b"}
May 25 13:06:44.182063 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.181998    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z7xmf" podStartSLOduration=68.011653057 podStartE2EDuration="1m11.181985255s" podCreationTimestamp="2026-05-25 13:05:33 +0000 UTC" firstStartedPulling="2026-05-25 13:06:39.853550193 +0000 UTC m=+66.572513310" lastFinishedPulling="2026-05-25 13:06:43.023882376 +0000 UTC m=+69.742845508" observedRunningTime="2026-05-25 13:06:44.181038379 +0000 UTC m=+70.900001520" watchObservedRunningTime="2026-05-25 13:06:44.181985255 +0000 UTC m=+70.900948395"
May 25 13:06:44.189990 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.189956    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9" event={"ID":"b57c7dbd-b155-4c76-bde0-22f1b87cb7de","Type":"ContainerStarted","Data":"a0a7c744f96ac31f8a1f41dacd8e746997a8ef04ab6b8ca4179d6f95284ec0c6"}
May 25 13:06:44.192828 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.192788    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f6lwr" event={"ID":"7844394a-a353-4e6f-9c29-75e111adcdb9","Type":"ContainerStarted","Data":"923bd7bc444bcd1127047a49ab987fd2cf2b31c1c76566adc922cf687efab9ea"}
May 25 13:06:44.192991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.192976    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f6lwr" event={"ID":"7844394a-a353-4e6f-9c29-75e111adcdb9","Type":"ContainerStarted","Data":"160ddb3cfe1ed4ce2f628525348aac7b573b2b422f07cb83d2000780c65098a4"}
May 25 13:06:44.194118 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.194103    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:44.195837 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.195819    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lftth" event={"ID":"95c09ce4-ed3a-4de9-baef-5b4e1fed5652","Type":"ContainerStarted","Data":"680c79c6c43c587300478ef9810e19093f70cfa21a4fb1ba0d0900d6b2f2255d"}
May 25 13:06:44.199312 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.199293    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bbd6764c5-zll4g" event={"ID":"b1f9b575-90ca-41e5-b2c0-c90137e06eac","Type":"ContainerStarted","Data":"3a1d4069fa8415d67d53adf0ed3e8864d594dc17a3d85d743819c15507f141e6"}
May 25 13:06:44.199464 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.199444    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bbd6764c5-zll4g" event={"ID":"b1f9b575-90ca-41e5-b2c0-c90137e06eac","Type":"ContainerStarted","Data":"cdb3b1d395ab6448157a0e43fac57b816f6ec3b78fc0fcdf5ceaaa1d77fbbddd"}
May 25 13:06:44.199692 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.199677    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:06:44.203988 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.202966    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tx5hr" podStartSLOduration=68.097059873 podStartE2EDuration="1m11.202950877s" podCreationTimestamp="2026-05-25 13:05:33 +0000 UTC" firstStartedPulling="2026-05-25 13:06:39.873841341 +0000 UTC m=+66.592804459" lastFinishedPulling="2026-05-25 13:06:42.979732343 +0000 UTC m=+69.698695463" observedRunningTime="2026-05-25 13:06:44.202056206 +0000 UTC m=+70.921019346" watchObservedRunningTime="2026-05-25 13:06:44.202950877 +0000 UTC m=+70.921914019"
May 25 13:06:44.204650 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.204627    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-rfml4" event={"ID":"a80cd8c7-074e-47ef-8de3-f2921430ad60","Type":"ContainerStarted","Data":"ba927e480a574ec5d40ec767e97499c5b863f13e3a92fab5efba8d0a35bbbd91"}
May 25 13:06:44.210201 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.210177    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n7bht" event={"ID":"989dfcac-2684-4ca2-929d-947ffe34fba1","Type":"ContainerStarted","Data":"86c4a406a55715c7dc375dac96dcfc684caf0c70392188c578ca203738adf993"}
May 25 13:06:44.210298 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.210210    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n7bht" event={"ID":"989dfcac-2684-4ca2-929d-947ffe34fba1","Type":"ContainerStarted","Data":"31937cebaa3f1a3aabc9c3231d06d957c993764dff4e0d3b1311fea583e6241b"}
May 25 13:06:44.210298 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.210225    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n7bht" event={"ID":"989dfcac-2684-4ca2-929d-947ffe34fba1","Type":"ContainerStarted","Data":"f03fdb9bfa7f92d0f953cb7a6db23da138369367537358c97339359232ea7a59"}
May 25 13:06:44.222591 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.222538    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-f6lwr" podStartSLOduration=34.246720974 podStartE2EDuration="38.222523754s" podCreationTimestamp="2026-05-25 13:06:06 +0000 UTC" firstStartedPulling="2026-05-25 13:06:38.982409717 +0000 UTC m=+65.701372834" lastFinishedPulling="2026-05-25 13:06:42.958212492 +0000 UTC m=+69.677175614" observedRunningTime="2026-05-25 13:06:44.220630559 +0000 UTC m=+70.939593700" watchObservedRunningTime="2026-05-25 13:06:44.222523754 +0000 UTC m=+70.941486896"
May 25 13:06:44.247207 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.247128    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lftth" podStartSLOduration=34.225240642 podStartE2EDuration="38.24710886s" podCreationTimestamp="2026-05-25 13:06:06 +0000 UTC" firstStartedPulling="2026-05-25 13:06:38.95786867 +0000 UTC m=+65.676831787" lastFinishedPulling="2026-05-25 13:06:42.979736875 +0000 UTC m=+69.698700005" observedRunningTime="2026-05-25 13:06:44.24552646 +0000 UTC m=+70.964489602" watchObservedRunningTime="2026-05-25 13:06:44.24710886 +0000 UTC m=+70.966071980"
May 25 13:06:44.270769 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:44.270190    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bbd6764c5-zll4g" podStartSLOduration=3.270172111 podStartE2EDuration="3.270172111s" podCreationTimestamp="2026-05-25 13:06:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:06:44.268068069 +0000 UTC m=+70.987031208" watchObservedRunningTime="2026-05-25 13:06:44.270172111 +0000 UTC m=+70.989135252"
May 25 13:06:51.241475 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.241436    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n7bht" event={"ID":"989dfcac-2684-4ca2-929d-947ffe34fba1","Type":"ContainerStarted","Data":"48f98c7dae2af4dfb5add0a07a84c47dca42e739cda807f0eec3840ab940300d"}
May 25 13:06:51.242957 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.242930    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7" event={"ID":"aaa33289-3ee5-4ad2-99a7-e2b158714252","Type":"ContainerStarted","Data":"e9fb6b8bb916322c7f0ed0728e6de773df6f3c8ff43fd882ea550e1f20ab43b5"}
May 25 13:06:51.243115 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.243098    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"
May 25 13:06:51.244565 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.244542    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7"
May 25 13:06:51.244665 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.244636    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cdcffffd8-vj5sd" event={"ID":"958511d0-c8a0-4f44-8698-57ad3b96851c","Type":"ContainerStarted","Data":"d6a529c7332ad82b0c98bb9d9fefd8adc9733e2d1b882c3ed18a15218537fadc"}
May 25 13:06:51.246377 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.246356    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9" event={"ID":"b57c7dbd-b155-4c76-bde0-22f1b87cb7de","Type":"ContainerStarted","Data":"9e6a850e027639352c2d0afb70455bca6e057e9c4678a09b07943e4a4927f55c"}
May 25 13:06:51.261127 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.260672    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n7bht" podStartSLOduration=2.740284829 podStartE2EDuration="10.260658552s" podCreationTimestamp="2026-05-25 13:06:41 +0000 UTC" firstStartedPulling="2026-05-25 13:06:43.411284829 +0000 UTC m=+70.130247959" lastFinishedPulling="2026-05-25 13:06:50.931658551 +0000 UTC m=+77.650621682" observedRunningTime="2026-05-25 13:06:51.258616016 +0000 UTC m=+77.977579157" watchObservedRunningTime="2026-05-25 13:06:51.260658552 +0000 UTC m=+77.979621692"
May 25 13:06:51.277360 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.277316    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7d8b8486dc-7lcj7" podStartSLOduration=2.6655011809999998 podStartE2EDuration="10.277307194s" podCreationTimestamp="2026-05-25 13:06:41 +0000 UTC" firstStartedPulling="2026-05-25 13:06:43.369528809 +0000 UTC m=+70.088491939" lastFinishedPulling="2026-05-25 13:06:50.981334819 +0000 UTC m=+77.700297952" observedRunningTime="2026-05-25 13:06:51.276075558 +0000 UTC m=+77.995038698" watchObservedRunningTime="2026-05-25 13:06:51.277307194 +0000 UTC m=+77.996270333"
May 25 13:06:51.292852 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.292675    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5cdcffffd8-vj5sd" podStartSLOduration=2.688369857 podStartE2EDuration="10.292658787s" podCreationTimestamp="2026-05-25 13:06:41 +0000 UTC" firstStartedPulling="2026-05-25 13:06:43.357930118 +0000 UTC m=+70.076893243" lastFinishedPulling="2026-05-25 13:06:50.962219056 +0000 UTC m=+77.681182173" observedRunningTime="2026-05-25 13:06:51.291587164 +0000 UTC m=+78.010550307" watchObservedRunningTime="2026-05-25 13:06:51.292658787 +0000 UTC m=+78.011621928"
May 25 13:06:51.305701 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.305648    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-bfc749cb4-98jb9" podStartSLOduration=2.686036755 podStartE2EDuration="10.305634212s" podCreationTimestamp="2026-05-25 13:06:41 +0000 UTC" firstStartedPulling="2026-05-25 13:06:43.339465705 +0000 UTC m=+70.058428840" lastFinishedPulling="2026-05-25 13:06:50.959063181 +0000 UTC m=+77.678026297" observedRunningTime="2026-05-25 13:06:51.30467348 +0000 UTC m=+78.023636619" watchObservedRunningTime="2026-05-25 13:06:51.305634212 +0000 UTC m=+78.024597371"
May 25 13:06:51.811096 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.811059    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:51.811096 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.811105    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:06:51.812611 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.812581    2559 patch_prober.go:28] interesting pod/console-5cdcffffd8-vj5sd container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.17:8443/health\": dial tcp 10.133.0.17:8443: connect: connection refused" start-of-body=
May 25 13:06:51.812757 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:51.812653    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5cdcffffd8-vj5sd" podUID="958511d0-c8a0-4f44-8698-57ad3b96851c" containerName="console" probeResult="failure" output="Get \"https://10.133.0.17:8443/health\": dial tcp 10.133.0.17:8443: connect: connection refused"
May 25 13:06:54.906653 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.906574    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5wpqv"]
May 25 13:06:54.914175 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.914148    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:54.916580 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.916554    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 25 13:06:54.917077 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.917062    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 25 13:06:54.917242 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.917162    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-47s74\""
May 25 13:06:54.917297 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.917265    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 25 13:06:54.917951 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.917570    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 25 13:06:54.917951 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.917633    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 25 13:06:54.917951 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.917823    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 25 13:06:54.994831 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.994342    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-wtmp\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:54.994831 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.994403    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/afba796a-339d-41ed-aaba-9f1aeece3b05-root\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:54.994831 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.994453    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-tls\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:54.994831 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.994481    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afba796a-339d-41ed-aaba-9f1aeece3b05-metrics-client-ca\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:54.994831 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.994512    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-accelerators-collector-config\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:54.994831 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.994542    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:54.994831 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.994583    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afba796a-339d-41ed-aaba-9f1aeece3b05-sys\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:54.994831 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.994610    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wvsc\" (UniqueName: \"kubernetes.io/projected/afba796a-339d-41ed-aaba-9f1aeece3b05-kube-api-access-8wvsc\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:54.994831 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:54.994647    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-textfile\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.095278 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.095237    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-textfile\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.095461 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.095305    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-wtmp\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.095461 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.095338    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/afba796a-339d-41ed-aaba-9f1aeece3b05-root\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.095461 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.095382    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-tls\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.095461 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.095412    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afba796a-339d-41ed-aaba-9f1aeece3b05-metrics-client-ca\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.095461 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.095441    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-accelerators-collector-config\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.095714 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.095466    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/afba796a-339d-41ed-aaba-9f1aeece3b05-root\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.095714 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.095470    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.095714 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.095516    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-wtmp\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.095714 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.095557    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afba796a-339d-41ed-aaba-9f1aeece3b05-sys\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.095714 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.095587    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8wvsc\" (UniqueName: \"kubernetes.io/projected/afba796a-339d-41ed-aaba-9f1aeece3b05-kube-api-access-8wvsc\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.095714 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.095617    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-textfile\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.095714 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.095652    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/afba796a-339d-41ed-aaba-9f1aeece3b05-sys\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.096227 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.096191    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-accelerators-collector-config\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.096310 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.096249    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afba796a-339d-41ed-aaba-9f1aeece3b05-metrics-client-ca\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.098155 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.098133    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.098338 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.098318    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/afba796a-339d-41ed-aaba-9f1aeece3b05-node-exporter-tls\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.105426 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.105384    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wvsc\" (UniqueName: \"kubernetes.io/projected/afba796a-339d-41ed-aaba-9f1aeece3b05-kube-api-access-8wvsc\") pod \"node-exporter-5wpqv\" (UID: \"afba796a-339d-41ed-aaba-9f1aeece3b05\") " pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.226573 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.226486    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5wpqv"
May 25 13:06:55.227595 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:55.227313    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f6lwr"
May 25 13:06:58.612886 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:06:58.612851    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cdcffffd8-vj5sd"]
May 25 13:07:00.803046 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:07:00.802993    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafba796a_339d_41ed_aaba_9f1aeece3b05.slice/crio-b3a3eff768154ef4beddb3e6569fa12389ab8420d14109995025608496c9ae99 WatchSource:0}: Error finding container b3a3eff768154ef4beddb3e6569fa12389ab8420d14109995025608496c9ae99: Status 404 returned error can't find the container with id b3a3eff768154ef4beddb3e6569fa12389ab8420d14109995025608496c9ae99
May 25 13:07:01.148991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.148922    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:07:01.276523 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.276490    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wpqv" event={"ID":"afba796a-339d-41ed-aaba-9f1aeece3b05","Type":"ContainerStarted","Data":"b3a3eff768154ef4beddb3e6569fa12389ab8420d14109995025608496c9ae99"}
May 25 13:07:01.284152 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.284114    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-54f547fb88-wv5hh"]
May 25 13:07:01.304115 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.304077    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54f547fb88-wv5hh"]
May 25 13:07:01.304259 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.304238    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.314114 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.313910    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
May 25 13:07:01.450915 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.450837    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-console-config\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.450915 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.450874    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blgv4\" (UniqueName: \"kubernetes.io/projected/3e195d1a-7856-4295-a725-e214ecba4acb-kube-api-access-blgv4\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.450915 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.450898    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-service-ca\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.451212 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.450940    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e195d1a-7856-4295-a725-e214ecba4acb-console-serving-cert\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.451212 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.451029    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-oauth-serving-cert\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.451212 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.451065    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e195d1a-7856-4295-a725-e214ecba4acb-console-oauth-config\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.451212 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.451100    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-trusted-ca-bundle\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.552554 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.552518    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-trusted-ca-bundle\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.552731 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.552578    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-console-config\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.552731 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.552601    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-blgv4\" (UniqueName: \"kubernetes.io/projected/3e195d1a-7856-4295-a725-e214ecba4acb-kube-api-access-blgv4\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.552731 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.552619    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-service-ca\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.552731 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.552637    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e195d1a-7856-4295-a725-e214ecba4acb-console-serving-cert\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.552731 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.552713    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-oauth-serving-cert\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.552931 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.552755    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e195d1a-7856-4295-a725-e214ecba4acb-console-oauth-config\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.553477 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.553424    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-service-ca\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.553601 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.553531    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-console-config\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.553661 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.553627    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-trusted-ca-bundle\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.553737 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.553653    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-oauth-serving-cert\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.555620 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.555587    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e195d1a-7856-4295-a725-e214ecba4acb-console-oauth-config\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.555822 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.555802    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e195d1a-7856-4295-a725-e214ecba4acb-console-serving-cert\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.561153 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.561131    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blgv4\" (UniqueName: \"kubernetes.io/projected/3e195d1a-7856-4295-a725-e214ecba4acb-kube-api-access-blgv4\") pod \"console-54f547fb88-wv5hh\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") " pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.617427 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.617198    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:01.650273 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.649963    2559 patch_prober.go:28] interesting pod/image-registry-bbd6764c5-zll4g container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 25 13:07:01.650273 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.650028    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-bbd6764c5-zll4g" podUID="b1f9b575-90ca-41e5-b2c0-c90137e06eac" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 25 13:07:01.924225 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:01.924185    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54f547fb88-wv5hh"]
May 25 13:07:02.122927 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:07:02.122851    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e195d1a_7856_4295_a725_e214ecba4acb.slice/crio-1541f89c4dfee580515a8ac6c0bb9476d89cb44c59bd872cf02c24f30d065598 WatchSource:0}: Error finding container 1541f89c4dfee580515a8ac6c0bb9476d89cb44c59bd872cf02c24f30d065598: Status 404 returned error can't find the container with id 1541f89c4dfee580515a8ac6c0bb9476d89cb44c59bd872cf02c24f30d065598
May 25 13:07:02.282587 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:02.282547    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-rfml4" event={"ID":"a80cd8c7-074e-47ef-8de3-f2921430ad60","Type":"ContainerStarted","Data":"91cf1f6a980d3673d901ef567c640aade89fd0db46de6d1ad29faa16ef30a0aa"}
May 25 13:07:02.282908 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:02.282876    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-57bd96b888-rfml4"
May 25 13:07:02.284356 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:02.284327    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f547fb88-wv5hh" event={"ID":"3e195d1a-7856-4295-a725-e214ecba4acb","Type":"ContainerStarted","Data":"1541f89c4dfee580515a8ac6c0bb9476d89cb44c59bd872cf02c24f30d065598"}
May 25 13:07:02.296299 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:02.296274    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-57bd96b888-rfml4"
May 25 13:07:02.321437 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:02.321377    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-57bd96b888-rfml4" podStartSLOduration=3.486858292 podStartE2EDuration="21.321357876s" podCreationTimestamp="2026-05-25 13:06:41 +0000 UTC" firstStartedPulling="2026-05-25 13:06:43.343708454 +0000 UTC m=+70.062671585" lastFinishedPulling="2026-05-25 13:07:01.178208041 +0000 UTC m=+87.897171169" observedRunningTime="2026-05-25 13:07:02.303819689 +0000 UTC m=+89.022782829" watchObservedRunningTime="2026-05-25 13:07:02.321357876 +0000 UTC m=+89.040321017"
May 25 13:07:03.290155 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:03.290112    2559 generic.go:358] "Generic (PLEG): container finished" podID="afba796a-339d-41ed-aaba-9f1aeece3b05" containerID="62acb1b9e87765c28c9edc0fda4127f3c7525b573cd6397e2b607783a4c2942b" exitCode=0
May 25 13:07:03.290761 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:03.290215    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wpqv" event={"ID":"afba796a-339d-41ed-aaba-9f1aeece3b05","Type":"ContainerDied","Data":"62acb1b9e87765c28c9edc0fda4127f3c7525b573cd6397e2b607783a4c2942b"}
May 25 13:07:03.292041 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:03.292014    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f547fb88-wv5hh" event={"ID":"3e195d1a-7856-4295-a725-e214ecba4acb","Type":"ContainerStarted","Data":"533dd3b219dff2c4ec490c970d0b5723cc68f13c89cef24ba79d67196f7ebc69"}
May 25 13:07:03.329456 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:03.329394    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-54f547fb88-wv5hh" podStartSLOduration=2.329374553 podStartE2EDuration="2.329374553s" podCreationTimestamp="2026-05-25 13:07:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:07:03.328761773 +0000 UTC m=+90.047724913" watchObservedRunningTime="2026-05-25 13:07:03.329374553 +0000 UTC m=+90.048337693"
May 25 13:07:04.298155 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:04.298110    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wpqv" event={"ID":"afba796a-339d-41ed-aaba-9f1aeece3b05","Type":"ContainerStarted","Data":"2a9bddcca864a3528706a1b6a40c923a39895bd14fc184385fce1a93819c6af1"}
May 25 13:07:04.298621 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:04.298162    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wpqv" event={"ID":"afba796a-339d-41ed-aaba-9f1aeece3b05","Type":"ContainerStarted","Data":"f50a44935a7219ca373a08fe7a1ed9995f44e3eac7b348d34233b74b6dc29925"}
May 25 13:07:04.318575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:04.318514    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5wpqv" podStartSLOduration=8.971510049 podStartE2EDuration="10.318500167s" podCreationTimestamp="2026-05-25 13:06:54 +0000 UTC" firstStartedPulling="2026-05-25 13:07:00.80532806 +0000 UTC m=+87.524291177" lastFinishedPulling="2026-05-25 13:07:02.152318163 +0000 UTC m=+88.871281295" observedRunningTime="2026-05-25 13:07:04.317804465 +0000 UTC m=+91.036767596" watchObservedRunningTime="2026-05-25 13:07:04.318500167 +0000 UTC m=+91.037463302"
May 25 13:07:05.226653 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:05.226616    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bbd6764c5-zll4g"
May 25 13:07:06.171294 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.171225    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-845c4b7766-sckdj" podUID="05ac39e7-0786-4cd3-9b66-05091c2c0c1a" containerName="registry" containerID="cri-o://3c21ba7adf47351843fbe9784e9e11577cabd6c9323da1c33b851b89ebc0e22a" gracePeriod=30
May 25 13:07:06.306713 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.306670    2559 generic.go:358] "Generic (PLEG): container finished" podID="05ac39e7-0786-4cd3-9b66-05091c2c0c1a" containerID="3c21ba7adf47351843fbe9784e9e11577cabd6c9323da1c33b851b89ebc0e22a" exitCode=0
May 25 13:07:06.306887 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.306760    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-845c4b7766-sckdj" event={"ID":"05ac39e7-0786-4cd3-9b66-05091c2c0c1a","Type":"ContainerDied","Data":"3c21ba7adf47351843fbe9784e9e11577cabd6c9323da1c33b851b89ebc0e22a"}
May 25 13:07:06.454336 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.454300    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:07:06.596919 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.596882    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-trusted-ca\") pod \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") "
May 25 13:07:06.597095 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.596998    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls\") pod \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") "
May 25 13:07:06.597095 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.597045    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-certificates\") pod \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") "
May 25 13:07:06.597095 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.597078    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-image-registry-private-configuration\") pod \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") "
May 25 13:07:06.597267 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.597115    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxljm\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-kube-api-access-lxljm\") pod \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") "
May 25 13:07:06.597267 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.597166    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-installation-pull-secrets\") pod \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") "
May 25 13:07:06.597267 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.597191    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-bound-sa-token\") pod \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") "
May 25 13:07:06.597267 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.597241    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-ca-trust-extracted\") pod \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\" (UID: \"05ac39e7-0786-4cd3-9b66-05091c2c0c1a\") "
May 25 13:07:06.597670 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.597632    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "05ac39e7-0786-4cd3-9b66-05091c2c0c1a" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:07:06.597898 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.597863    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "05ac39e7-0786-4cd3-9b66-05091c2c0c1a" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:07:06.600085 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.600049    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "05ac39e7-0786-4cd3-9b66-05091c2c0c1a" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:07:06.600180 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.600053    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "05ac39e7-0786-4cd3-9b66-05091c2c0c1a" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:07:06.600665 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.600627    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-kube-api-access-lxljm" (OuterVolumeSpecName: "kube-api-access-lxljm") pod "05ac39e7-0786-4cd3-9b66-05091c2c0c1a" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a"). InnerVolumeSpecName "kube-api-access-lxljm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:07:06.601008 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.600965    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "05ac39e7-0786-4cd3-9b66-05091c2c0c1a" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:07:06.601095 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.601025    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "05ac39e7-0786-4cd3-9b66-05091c2c0c1a" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:07:06.608073 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.608044    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "05ac39e7-0786-4cd3-9b66-05091c2c0c1a" (UID: "05ac39e7-0786-4cd3-9b66-05091c2c0c1a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:07:06.698527 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.698444    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxljm\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-kube-api-access-lxljm\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:06.698527 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.698479    2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-installation-pull-secrets\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:06.698527 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.698489    2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-bound-sa-token\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:06.698527 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.698501    2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-ca-trust-extracted\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:06.698527 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.698515    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-trusted-ca\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:06.698527 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.698526    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-tls\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:06.698886 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.698541    2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-registry-certificates\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:06.698886 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:06.698557    2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05ac39e7-0786-4cd3-9b66-05091c2c0c1a-image-registry-private-configuration\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:07.312247 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:07.312212    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-845c4b7766-sckdj" event={"ID":"05ac39e7-0786-4cd3-9b66-05091c2c0c1a","Type":"ContainerDied","Data":"d1f1883a8aa62d125548bf6798b59cd06911816ad2f4f1ffab4500018b69762b"}
May 25 13:07:07.312714 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:07.312264    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-845c4b7766-sckdj"
May 25 13:07:07.312714 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:07.312270    2559 scope.go:117] "RemoveContainer" containerID="3c21ba7adf47351843fbe9784e9e11577cabd6c9323da1c33b851b89ebc0e22a"
May 25 13:07:07.336781 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:07.336750    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-845c4b7766-sckdj"]
May 25 13:07:07.340384 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:07.340362    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-845c4b7766-sckdj"]
May 25 13:07:07.869070 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:07.869043    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05ac39e7-0786-4cd3-9b66-05091c2c0c1a" path="/var/lib/kubelet/pods/05ac39e7-0786-4cd3-9b66-05091c2c0c1a/volumes"
May 25 13:07:11.617926 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:11.617891    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:11.617926 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:11.617929    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:11.622636 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:11.622607    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:12.331886 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:12.331857    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:07:16.227098 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:16.227068    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z7xmf"
May 25 13:07:22.304426 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:22.304392    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lftth_95c09ce4-ed3a-4de9-baef-5b4e1fed5652/serve-healthcheck-canary/0.log"
May 25 13:07:23.634669 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:23.634626    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-5cdcffffd8-vj5sd" podUID="958511d0-c8a0-4f44-8698-57ad3b96851c" containerName="console" containerID="cri-o://d6a529c7332ad82b0c98bb9d9fefd8adc9733e2d1b882c3ed18a15218537fadc" gracePeriod=15
May 25 13:07:23.932175 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:23.932147    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5cdcffffd8-vj5sd_958511d0-c8a0-4f44-8698-57ad3b96851c/console/0.log"
May 25 13:07:23.932321 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:23.932217    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:07:24.043703 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.043671    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-console-config\") pod \"958511d0-c8a0-4f44-8698-57ad3b96851c\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") "
May 25 13:07:24.043906 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.043748    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/958511d0-c8a0-4f44-8698-57ad3b96851c-console-oauth-config\") pod \"958511d0-c8a0-4f44-8698-57ad3b96851c\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") "
May 25 13:07:24.043906 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.043775    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrsd5\" (UniqueName: \"kubernetes.io/projected/958511d0-c8a0-4f44-8698-57ad3b96851c-kube-api-access-xrsd5\") pod \"958511d0-c8a0-4f44-8698-57ad3b96851c\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") "
May 25 13:07:24.043906 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.043817    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-oauth-serving-cert\") pod \"958511d0-c8a0-4f44-8698-57ad3b96851c\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") "
May 25 13:07:24.043906 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.043842    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-service-ca\") pod \"958511d0-c8a0-4f44-8698-57ad3b96851c\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") "
May 25 13:07:24.043906 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.043878    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/958511d0-c8a0-4f44-8698-57ad3b96851c-console-serving-cert\") pod \"958511d0-c8a0-4f44-8698-57ad3b96851c\" (UID: \"958511d0-c8a0-4f44-8698-57ad3b96851c\") "
May 25 13:07:24.044182 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.044147    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-console-config" (OuterVolumeSpecName: "console-config") pod "958511d0-c8a0-4f44-8698-57ad3b96851c" (UID: "958511d0-c8a0-4f44-8698-57ad3b96851c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:07:24.044262 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.044230    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "958511d0-c8a0-4f44-8698-57ad3b96851c" (UID: "958511d0-c8a0-4f44-8698-57ad3b96851c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:07:24.044313 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.044266    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-service-ca" (OuterVolumeSpecName: "service-ca") pod "958511d0-c8a0-4f44-8698-57ad3b96851c" (UID: "958511d0-c8a0-4f44-8698-57ad3b96851c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:07:24.046127 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.046098    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/958511d0-c8a0-4f44-8698-57ad3b96851c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "958511d0-c8a0-4f44-8698-57ad3b96851c" (UID: "958511d0-c8a0-4f44-8698-57ad3b96851c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:07:24.046249 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.046156    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/958511d0-c8a0-4f44-8698-57ad3b96851c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "958511d0-c8a0-4f44-8698-57ad3b96851c" (UID: "958511d0-c8a0-4f44-8698-57ad3b96851c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:07:24.046249 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.046202    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/958511d0-c8a0-4f44-8698-57ad3b96851c-kube-api-access-xrsd5" (OuterVolumeSpecName: "kube-api-access-xrsd5") pod "958511d0-c8a0-4f44-8698-57ad3b96851c" (UID: "958511d0-c8a0-4f44-8698-57ad3b96851c"). InnerVolumeSpecName "kube-api-access-xrsd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:07:24.145250 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.145212    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/958511d0-c8a0-4f44-8698-57ad3b96851c-console-oauth-config\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:24.145250 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.145244    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrsd5\" (UniqueName: \"kubernetes.io/projected/958511d0-c8a0-4f44-8698-57ad3b96851c-kube-api-access-xrsd5\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:24.145250 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.145256    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-oauth-serving-cert\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:24.145483 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.145266    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-service-ca\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:24.145483 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.145276    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/958511d0-c8a0-4f44-8698-57ad3b96851c-console-serving-cert\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:24.145483 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.145284    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/958511d0-c8a0-4f44-8698-57ad3b96851c-console-config\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:07:24.361540 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.361517    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5cdcffffd8-vj5sd_958511d0-c8a0-4f44-8698-57ad3b96851c/console/0.log"
May 25 13:07:24.361704 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.361553    2559 generic.go:358] "Generic (PLEG): container finished" podID="958511d0-c8a0-4f44-8698-57ad3b96851c" containerID="d6a529c7332ad82b0c98bb9d9fefd8adc9733e2d1b882c3ed18a15218537fadc" exitCode=2
May 25 13:07:24.361704 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.361585    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cdcffffd8-vj5sd" event={"ID":"958511d0-c8a0-4f44-8698-57ad3b96851c","Type":"ContainerDied","Data":"d6a529c7332ad82b0c98bb9d9fefd8adc9733e2d1b882c3ed18a15218537fadc"}
May 25 13:07:24.361704 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.361607    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cdcffffd8-vj5sd" event={"ID":"958511d0-c8a0-4f44-8698-57ad3b96851c","Type":"ContainerDied","Data":"d1a55e61a380ccd00ea4511b4ceb3619804a8be9a653bad58237d9ae45806f3b"}
May 25 13:07:24.361704 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.361619    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cdcffffd8-vj5sd"
May 25 13:07:24.361704 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.361625    2559 scope.go:117] "RemoveContainer" containerID="d6a529c7332ad82b0c98bb9d9fefd8adc9733e2d1b882c3ed18a15218537fadc"
May 25 13:07:24.371567 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.370351    2559 scope.go:117] "RemoveContainer" containerID="d6a529c7332ad82b0c98bb9d9fefd8adc9733e2d1b882c3ed18a15218537fadc"
May 25 13:07:24.372033 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:07:24.371998    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6a529c7332ad82b0c98bb9d9fefd8adc9733e2d1b882c3ed18a15218537fadc\": container with ID starting with d6a529c7332ad82b0c98bb9d9fefd8adc9733e2d1b882c3ed18a15218537fadc not found: ID does not exist" containerID="d6a529c7332ad82b0c98bb9d9fefd8adc9733e2d1b882c3ed18a15218537fadc"
May 25 13:07:24.372139 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.372045    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6a529c7332ad82b0c98bb9d9fefd8adc9733e2d1b882c3ed18a15218537fadc"} err="failed to get container status \"d6a529c7332ad82b0c98bb9d9fefd8adc9733e2d1b882c3ed18a15218537fadc\": rpc error: code = NotFound desc = could not find container \"d6a529c7332ad82b0c98bb9d9fefd8adc9733e2d1b882c3ed18a15218537fadc\": container with ID starting with d6a529c7332ad82b0c98bb9d9fefd8adc9733e2d1b882c3ed18a15218537fadc not found: ID does not exist"
May 25 13:07:24.385095 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.385064    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cdcffffd8-vj5sd"]
May 25 13:07:24.390457 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:24.390427    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cdcffffd8-vj5sd"]
May 25 13:07:25.868594 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:07:25.868562    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="958511d0-c8a0-4f44-8698-57ad3b96851c" path="/var/lib/kubelet/pods/958511d0-c8a0-4f44-8698-57ad3b96851c/volumes"
May 25 13:08:22.457316 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.457235    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6885bcc954-z2fnb"]
May 25 13:08:22.457756 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.457484    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05ac39e7-0786-4cd3-9b66-05091c2c0c1a" containerName="registry"
May 25 13:08:22.457756 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.457495    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="05ac39e7-0786-4cd3-9b66-05091c2c0c1a" containerName="registry"
May 25 13:08:22.457756 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.457506    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="958511d0-c8a0-4f44-8698-57ad3b96851c" containerName="console"
May 25 13:08:22.457756 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.457511    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="958511d0-c8a0-4f44-8698-57ad3b96851c" containerName="console"
May 25 13:08:22.457756 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.457558    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="05ac39e7-0786-4cd3-9b66-05091c2c0c1a" containerName="registry"
May 25 13:08:22.457756 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.457566    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="958511d0-c8a0-4f44-8698-57ad3b96851c" containerName="console"
May 25 13:08:22.459423 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.459406    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.473677 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.473651    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6885bcc954-z2fnb"]
May 25 13:08:22.561755 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.561705    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-service-ca\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.561913 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.561763    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-config\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.561913 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.561829    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-trusted-ca-bundle\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.561913 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.561905    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-serving-cert\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.562023 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.561941    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj8t8\" (UniqueName: \"kubernetes.io/projected/bb56f02c-7b40-4edd-9bd5-251348df9bbb-kube-api-access-pj8t8\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.562023 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.561967    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-oauth-config\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.562089 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.562042    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-oauth-serving-cert\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.662553 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.662515    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-serving-cert\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.662553 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.662553    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pj8t8\" (UniqueName: \"kubernetes.io/projected/bb56f02c-7b40-4edd-9bd5-251348df9bbb-kube-api-access-pj8t8\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.662815 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.662576    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-oauth-config\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.662815 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.662609    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-oauth-serving-cert\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.662815 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.662633    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-service-ca\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.662815 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.662652    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-config\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.662815 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.662672    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-trusted-ca-bundle\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.663355 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.663325    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-oauth-serving-cert\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.663486 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.663466    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-config\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.663533 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.663466    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-service-ca\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.663982 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.663958    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-trusted-ca-bundle\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.665075 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.665054    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-oauth-config\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.665163 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.665144    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-serving-cert\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.673840 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.673818    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj8t8\" (UniqueName: \"kubernetes.io/projected/bb56f02c-7b40-4edd-9bd5-251348df9bbb-kube-api-access-pj8t8\") pod \"console-6885bcc954-z2fnb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") " pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.768614 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.768575    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:22.890337 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:22.890305    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6885bcc954-z2fnb"]
May 25 13:08:22.893362 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:08:22.893335    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb56f02c_7b40_4edd_9bd5_251348df9bbb.slice/crio-b2cc8d3c73afff067aa6346c2f865f991c8bd0ec23543211e9a536ca0cda0efc WatchSource:0}: Error finding container b2cc8d3c73afff067aa6346c2f865f991c8bd0ec23543211e9a536ca0cda0efc: Status 404 returned error can't find the container with id b2cc8d3c73afff067aa6346c2f865f991c8bd0ec23543211e9a536ca0cda0efc
May 25 13:08:23.518401 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:23.518362    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6885bcc954-z2fnb" event={"ID":"bb56f02c-7b40-4edd-9bd5-251348df9bbb","Type":"ContainerStarted","Data":"63f7bd445aa0313a293f6e2d5d380e5e787b8a1833b6d98bb1a2c299436825de"}
May 25 13:08:23.518401 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:23.518407    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6885bcc954-z2fnb" event={"ID":"bb56f02c-7b40-4edd-9bd5-251348df9bbb","Type":"ContainerStarted","Data":"b2cc8d3c73afff067aa6346c2f865f991c8bd0ec23543211e9a536ca0cda0efc"}
May 25 13:08:23.536021 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:23.535938    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6885bcc954-z2fnb" podStartSLOduration=1.535920803 podStartE2EDuration="1.535920803s" podCreationTimestamp="2026-05-25 13:08:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:08:23.535407058 +0000 UTC m=+170.254370208" watchObservedRunningTime="2026-05-25 13:08:23.535920803 +0000 UTC m=+170.254883942"
May 25 13:08:32.769610 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:32.769561    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:32.769610 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:32.769623    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:32.775118 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:32.775097    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:33.545515 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.545483    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:08:33.599804 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.599773    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54f547fb88-wv5hh"]
May 25 13:08:33.732778 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.732745    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f7447dd6b-vlp6b"]
May 25 13:08:33.734977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.734960    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.757148 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.757119    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f7447dd6b-vlp6b"]
May 25 13:08:33.850833 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.850754    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-service-ca\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.850833 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.850784    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q424k\" (UniqueName: \"kubernetes.io/projected/d21b804b-2f92-4980-9c6b-bf60a0c583d0-kube-api-access-q424k\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.850833 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.850813    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-config\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.851251 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.850892    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-oauth-serving-cert\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.851251 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.850917    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-oauth-config\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.851251 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.850938    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-serving-cert\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.851251 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.850996    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-trusted-ca-bundle\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.951891 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.951854    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-config\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.952096 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.951909    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-oauth-serving-cert\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.952096 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.951928    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-oauth-config\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.952096 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.951945    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-serving-cert\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.952096 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.952072    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-trusted-ca-bundle\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.952314 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.952142    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-service-ca\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.952314 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.952168    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q424k\" (UniqueName: \"kubernetes.io/projected/d21b804b-2f92-4980-9c6b-bf60a0c583d0-kube-api-access-q424k\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.952714 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.952687    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-oauth-serving-cert\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.952826 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.952803    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-config\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.952864 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.952827    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-service-ca\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.953066 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.953039    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-trusted-ca-bundle\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.954388 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.954366    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-oauth-config\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.954503 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.954486    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-serving-cert\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:33.961571 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:33.961551    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q424k\" (UniqueName: \"kubernetes.io/projected/d21b804b-2f92-4980-9c6b-bf60a0c583d0-kube-api-access-q424k\") pod \"console-7f7447dd6b-vlp6b\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") " pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:34.043551 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:34.043507    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:34.168437 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:34.168401    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f7447dd6b-vlp6b"]
May 25 13:08:34.171242 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:08:34.171215    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21b804b_2f92_4980_9c6b_bf60a0c583d0.slice/crio-65455599b371fc462b3e14ec9d45f5c24421017fe49e68f39092f467de803730 WatchSource:0}: Error finding container 65455599b371fc462b3e14ec9d45f5c24421017fe49e68f39092f467de803730: Status 404 returned error can't find the container with id 65455599b371fc462b3e14ec9d45f5c24421017fe49e68f39092f467de803730
May 25 13:08:34.546608 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:34.546566    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7447dd6b-vlp6b" event={"ID":"d21b804b-2f92-4980-9c6b-bf60a0c583d0","Type":"ContainerStarted","Data":"d8992215fc855cc7f7dd807832177b477a9524066b6637294ac4f0257aa20eb1"}
May 25 13:08:34.546608 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:34.546613    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7447dd6b-vlp6b" event={"ID":"d21b804b-2f92-4980-9c6b-bf60a0c583d0","Type":"ContainerStarted","Data":"65455599b371fc462b3e14ec9d45f5c24421017fe49e68f39092f467de803730"}
May 25 13:08:34.564113 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:34.564061    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7f7447dd6b-vlp6b" podStartSLOduration=1.5640460699999998 podStartE2EDuration="1.56404607s" podCreationTimestamp="2026-05-25 13:08:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:08:34.563874052 +0000 UTC m=+181.282837213" watchObservedRunningTime="2026-05-25 13:08:34.56404607 +0000 UTC m=+181.283009210"
May 25 13:08:44.044381 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:44.044346    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:44.044381 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:44.044387    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:44.049007 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:44.048983    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:44.575993 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:44.575966    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:08:44.620692 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:44.620661    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6885bcc954-z2fnb"]
May 25 13:08:58.625454 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.625415    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-54f547fb88-wv5hh" podUID="3e195d1a-7856-4295-a725-e214ecba4acb" containerName="console" containerID="cri-o://533dd3b219dff2c4ec490c970d0b5723cc68f13c89cef24ba79d67196f7ebc69" gracePeriod=15
May 25 13:08:58.853997 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.853974    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-54f547fb88-wv5hh_3e195d1a-7856-4295-a725-e214ecba4acb/console/0.log"
May 25 13:08:58.854113 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.854033    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:08:58.952628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.952542    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blgv4\" (UniqueName: \"kubernetes.io/projected/3e195d1a-7856-4295-a725-e214ecba4acb-kube-api-access-blgv4\") pod \"3e195d1a-7856-4295-a725-e214ecba4acb\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") "
May 25 13:08:58.952628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.952579    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e195d1a-7856-4295-a725-e214ecba4acb-console-oauth-config\") pod \"3e195d1a-7856-4295-a725-e214ecba4acb\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") "
May 25 13:08:58.952628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.952603    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-trusted-ca-bundle\") pod \"3e195d1a-7856-4295-a725-e214ecba4acb\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") "
May 25 13:08:58.952922 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.952741    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-service-ca\") pod \"3e195d1a-7856-4295-a725-e214ecba4acb\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") "
May 25 13:08:58.952922 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.952789    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-oauth-serving-cert\") pod \"3e195d1a-7856-4295-a725-e214ecba4acb\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") "
May 25 13:08:58.952922 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.952826    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-console-config\") pod \"3e195d1a-7856-4295-a725-e214ecba4acb\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") "
May 25 13:08:58.952922 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.952850    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e195d1a-7856-4295-a725-e214ecba4acb-console-serving-cert\") pod \"3e195d1a-7856-4295-a725-e214ecba4acb\" (UID: \"3e195d1a-7856-4295-a725-e214ecba4acb\") "
May 25 13:08:58.953165 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.953131    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3e195d1a-7856-4295-a725-e214ecba4acb" (UID: "3e195d1a-7856-4295-a725-e214ecba4acb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:08:58.953240 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.953195    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-service-ca" (OuterVolumeSpecName: "service-ca") pod "3e195d1a-7856-4295-a725-e214ecba4acb" (UID: "3e195d1a-7856-4295-a725-e214ecba4acb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:08:58.953240 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.953214    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3e195d1a-7856-4295-a725-e214ecba4acb" (UID: "3e195d1a-7856-4295-a725-e214ecba4acb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:08:58.953240 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.953224    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-console-config" (OuterVolumeSpecName: "console-config") pod "3e195d1a-7856-4295-a725-e214ecba4acb" (UID: "3e195d1a-7856-4295-a725-e214ecba4acb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:08:58.954815 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.954794    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e195d1a-7856-4295-a725-e214ecba4acb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3e195d1a-7856-4295-a725-e214ecba4acb" (UID: "3e195d1a-7856-4295-a725-e214ecba4acb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:08:58.954894 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.954827    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e195d1a-7856-4295-a725-e214ecba4acb-kube-api-access-blgv4" (OuterVolumeSpecName: "kube-api-access-blgv4") pod "3e195d1a-7856-4295-a725-e214ecba4acb" (UID: "3e195d1a-7856-4295-a725-e214ecba4acb"). InnerVolumeSpecName "kube-api-access-blgv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:08:58.955069 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:58.955050    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e195d1a-7856-4295-a725-e214ecba4acb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3e195d1a-7856-4295-a725-e214ecba4acb" (UID: "3e195d1a-7856-4295-a725-e214ecba4acb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:08:59.053622 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.053570    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-console-config\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:08:59.053622 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.053616    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e195d1a-7856-4295-a725-e214ecba4acb-console-serving-cert\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:08:59.053622 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.053631    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blgv4\" (UniqueName: \"kubernetes.io/projected/3e195d1a-7856-4295-a725-e214ecba4acb-kube-api-access-blgv4\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:08:59.053946 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.053648    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e195d1a-7856-4295-a725-e214ecba4acb-console-oauth-config\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:08:59.053946 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.053660    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-trusted-ca-bundle\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:08:59.053946 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.053672    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-service-ca\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:08:59.053946 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.053683    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e195d1a-7856-4295-a725-e214ecba4acb-oauth-serving-cert\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:08:59.615830 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.615804    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-54f547fb88-wv5hh_3e195d1a-7856-4295-a725-e214ecba4acb/console/0.log"
May 25 13:08:59.615991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.615844    2559 generic.go:358] "Generic (PLEG): container finished" podID="3e195d1a-7856-4295-a725-e214ecba4acb" containerID="533dd3b219dff2c4ec490c970d0b5723cc68f13c89cef24ba79d67196f7ebc69" exitCode=2
May 25 13:08:59.615991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.615904    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f547fb88-wv5hh" event={"ID":"3e195d1a-7856-4295-a725-e214ecba4acb","Type":"ContainerDied","Data":"533dd3b219dff2c4ec490c970d0b5723cc68f13c89cef24ba79d67196f7ebc69"}
May 25 13:08:59.615991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.615916    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f547fb88-wv5hh"
May 25 13:08:59.615991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.615937    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f547fb88-wv5hh" event={"ID":"3e195d1a-7856-4295-a725-e214ecba4acb","Type":"ContainerDied","Data":"1541f89c4dfee580515a8ac6c0bb9476d89cb44c59bd872cf02c24f30d065598"}
May 25 13:08:59.615991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.615958    2559 scope.go:117] "RemoveContainer" containerID="533dd3b219dff2c4ec490c970d0b5723cc68f13c89cef24ba79d67196f7ebc69"
May 25 13:08:59.623801 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.623781    2559 scope.go:117] "RemoveContainer" containerID="533dd3b219dff2c4ec490c970d0b5723cc68f13c89cef24ba79d67196f7ebc69"
May 25 13:08:59.624052 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:08:59.624027    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"533dd3b219dff2c4ec490c970d0b5723cc68f13c89cef24ba79d67196f7ebc69\": container with ID starting with 533dd3b219dff2c4ec490c970d0b5723cc68f13c89cef24ba79d67196f7ebc69 not found: ID does not exist" containerID="533dd3b219dff2c4ec490c970d0b5723cc68f13c89cef24ba79d67196f7ebc69"
May 25 13:08:59.624099 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.624065    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"533dd3b219dff2c4ec490c970d0b5723cc68f13c89cef24ba79d67196f7ebc69"} err="failed to get container status \"533dd3b219dff2c4ec490c970d0b5723cc68f13c89cef24ba79d67196f7ebc69\": rpc error: code = NotFound desc = could not find container \"533dd3b219dff2c4ec490c970d0b5723cc68f13c89cef24ba79d67196f7ebc69\": container with ID starting with 533dd3b219dff2c4ec490c970d0b5723cc68f13c89cef24ba79d67196f7ebc69 not found: ID does not exist"
May 25 13:08:59.636396 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.636374    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54f547fb88-wv5hh"]
May 25 13:08:59.642123 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.642101    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54f547fb88-wv5hh"]
May 25 13:08:59.868935 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:08:59.868865    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e195d1a-7856-4295-a725-e214ecba4acb" path="/var/lib/kubelet/pods/3e195d1a-7856-4295-a725-e214ecba4acb/volumes"
May 25 13:09:09.639621 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.639551    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-6885bcc954-z2fnb" podUID="bb56f02c-7b40-4edd-9bd5-251348df9bbb" containerName="console" containerID="cri-o://63f7bd445aa0313a293f6e2d5d380e5e787b8a1833b6d98bb1a2c299436825de" gracePeriod=15
May 25 13:09:09.878331 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.878311    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6885bcc954-z2fnb_bb56f02c-7b40-4edd-9bd5-251348df9bbb/console/0.log"
May 25 13:09:09.878443 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.878370    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:09:09.922489 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.922419    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-config\") pod \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") "
May 25 13:09:09.922489 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.922456    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-oauth-config\") pod \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") "
May 25 13:09:09.922489 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.922488    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-serving-cert\") pod \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") "
May 25 13:09:09.922709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.922601    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-oauth-serving-cert\") pod \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") "
May 25 13:09:09.922709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.922652    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-trusted-ca-bundle\") pod \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") "
May 25 13:09:09.922920 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.922884    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj8t8\" (UniqueName: \"kubernetes.io/projected/bb56f02c-7b40-4edd-9bd5-251348df9bbb-kube-api-access-pj8t8\") pod \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") "
May 25 13:09:09.922992 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.922915    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-config" (OuterVolumeSpecName: "console-config") pod "bb56f02c-7b40-4edd-9bd5-251348df9bbb" (UID: "bb56f02c-7b40-4edd-9bd5-251348df9bbb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:09:09.922992 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.922936    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-service-ca\") pod \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\" (UID: \"bb56f02c-7b40-4edd-9bd5-251348df9bbb\") "
May 25 13:09:09.923092 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.923001    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bb56f02c-7b40-4edd-9bd5-251348df9bbb" (UID: "bb56f02c-7b40-4edd-9bd5-251348df9bbb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:09:09.923092 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.922994    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bb56f02c-7b40-4edd-9bd5-251348df9bbb" (UID: "bb56f02c-7b40-4edd-9bd5-251348df9bbb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:09:09.923191 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.923169    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-oauth-serving-cert\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:09:09.923191 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.923187    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-trusted-ca-bundle\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:09:09.923288 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.923203    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-config\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:09:09.923340 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.923295    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-service-ca" (OuterVolumeSpecName: "service-ca") pod "bb56f02c-7b40-4edd-9bd5-251348df9bbb" (UID: "bb56f02c-7b40-4edd-9bd5-251348df9bbb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:09:09.924701 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.924679    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bb56f02c-7b40-4edd-9bd5-251348df9bbb" (UID: "bb56f02c-7b40-4edd-9bd5-251348df9bbb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:09:09.924826 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.924798    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bb56f02c-7b40-4edd-9bd5-251348df9bbb" (UID: "bb56f02c-7b40-4edd-9bd5-251348df9bbb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:09:09.924886 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:09.924859    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb56f02c-7b40-4edd-9bd5-251348df9bbb-kube-api-access-pj8t8" (OuterVolumeSpecName: "kube-api-access-pj8t8") pod "bb56f02c-7b40-4edd-9bd5-251348df9bbb" (UID: "bb56f02c-7b40-4edd-9bd5-251348df9bbb"). InnerVolumeSpecName "kube-api-access-pj8t8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:09:10.023495 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.023463    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-oauth-config\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:09:10.023495 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.023491    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb56f02c-7b40-4edd-9bd5-251348df9bbb-console-serving-cert\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:09:10.023495 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.023501    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj8t8\" (UniqueName: \"kubernetes.io/projected/bb56f02c-7b40-4edd-9bd5-251348df9bbb-kube-api-access-pj8t8\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:09:10.023699 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.023511    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb56f02c-7b40-4edd-9bd5-251348df9bbb-service-ca\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:09:10.648877 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.648853    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6885bcc954-z2fnb_bb56f02c-7b40-4edd-9bd5-251348df9bbb/console/0.log"
May 25 13:09:10.649306 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.648894    2559 generic.go:358] "Generic (PLEG): container finished" podID="bb56f02c-7b40-4edd-9bd5-251348df9bbb" containerID="63f7bd445aa0313a293f6e2d5d380e5e787b8a1833b6d98bb1a2c299436825de" exitCode=2
May 25 13:09:10.649306 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.648965    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6885bcc954-z2fnb"
May 25 13:09:10.649306 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.648980    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6885bcc954-z2fnb" event={"ID":"bb56f02c-7b40-4edd-9bd5-251348df9bbb","Type":"ContainerDied","Data":"63f7bd445aa0313a293f6e2d5d380e5e787b8a1833b6d98bb1a2c299436825de"}
May 25 13:09:10.649306 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.649019    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6885bcc954-z2fnb" event={"ID":"bb56f02c-7b40-4edd-9bd5-251348df9bbb","Type":"ContainerDied","Data":"b2cc8d3c73afff067aa6346c2f865f991c8bd0ec23543211e9a536ca0cda0efc"}
May 25 13:09:10.649306 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.649035    2559 scope.go:117] "RemoveContainer" containerID="63f7bd445aa0313a293f6e2d5d380e5e787b8a1833b6d98bb1a2c299436825de"
May 25 13:09:10.657463 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.657343    2559 scope.go:117] "RemoveContainer" containerID="63f7bd445aa0313a293f6e2d5d380e5e787b8a1833b6d98bb1a2c299436825de"
May 25 13:09:10.657661 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:09:10.657641    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63f7bd445aa0313a293f6e2d5d380e5e787b8a1833b6d98bb1a2c299436825de\": container with ID starting with 63f7bd445aa0313a293f6e2d5d380e5e787b8a1833b6d98bb1a2c299436825de not found: ID does not exist" containerID="63f7bd445aa0313a293f6e2d5d380e5e787b8a1833b6d98bb1a2c299436825de"
May 25 13:09:10.657712 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.657669    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63f7bd445aa0313a293f6e2d5d380e5e787b8a1833b6d98bb1a2c299436825de"} err="failed to get container status \"63f7bd445aa0313a293f6e2d5d380e5e787b8a1833b6d98bb1a2c299436825de\": rpc error: code = NotFound desc = could not find container \"63f7bd445aa0313a293f6e2d5d380e5e787b8a1833b6d98bb1a2c299436825de\": container with ID starting with 63f7bd445aa0313a293f6e2d5d380e5e787b8a1833b6d98bb1a2c299436825de not found: ID does not exist"
May 25 13:09:10.671114 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.671089    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6885bcc954-z2fnb"]
May 25 13:09:10.674198 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:10.674178    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6885bcc954-z2fnb"]
May 25 13:09:11.868671 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:09:11.868643    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb56f02c-7b40-4edd-9bd5-251348df9bbb" path="/var/lib/kubelet/pods/bb56f02c-7b40-4edd-9bd5-251348df9bbb/volumes"
May 25 13:10:33.747788 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:10:33.747756    2559 kubelet.go:1628] "Image garbage collection succeeded"
May 25 13:13:28.446913 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.446879    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cc6bf9c98-s9zcq"]
May 25 13:13:28.447371 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.447121    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e195d1a-7856-4295-a725-e214ecba4acb" containerName="console"
May 25 13:13:28.447371 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.447132    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e195d1a-7856-4295-a725-e214ecba4acb" containerName="console"
May 25 13:13:28.447371 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.447141    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb56f02c-7b40-4edd-9bd5-251348df9bbb" containerName="console"
May 25 13:13:28.447371 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.447147    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb56f02c-7b40-4edd-9bd5-251348df9bbb" containerName="console"
May 25 13:13:28.447371 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.447186    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="3e195d1a-7856-4295-a725-e214ecba4acb" containerName="console"
May 25 13:13:28.447371 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.447194    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="bb56f02c-7b40-4edd-9bd5-251348df9bbb" containerName="console"
May 25 13:13:28.448906 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.448888    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.500461 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.500429    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cc6bf9c98-s9zcq"]
May 25 13:13:28.562139 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.562103    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86e8b3d0-df94-4f49-acf7-702079789dcb-console-config\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.562139 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.562149    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86e8b3d0-df94-4f49-acf7-702079789dcb-trusted-ca-bundle\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.562354 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.562172    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86e8b3d0-df94-4f49-acf7-702079789dcb-oauth-serving-cert\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.562354 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.562279    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86e8b3d0-df94-4f49-acf7-702079789dcb-console-oauth-config\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.562354 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.562309    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86e8b3d0-df94-4f49-acf7-702079789dcb-service-ca\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.562354 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.562330    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q72m9\" (UniqueName: \"kubernetes.io/projected/86e8b3d0-df94-4f49-acf7-702079789dcb-kube-api-access-q72m9\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.562479 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.562373    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86e8b3d0-df94-4f49-acf7-702079789dcb-console-serving-cert\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.663197 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.663147    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86e8b3d0-df94-4f49-acf7-702079789dcb-console-config\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.663197 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.663203    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86e8b3d0-df94-4f49-acf7-702079789dcb-trusted-ca-bundle\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.663370 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.663302    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86e8b3d0-df94-4f49-acf7-702079789dcb-oauth-serving-cert\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.663410 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.663373    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86e8b3d0-df94-4f49-acf7-702079789dcb-console-oauth-config\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.663410 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.663391    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86e8b3d0-df94-4f49-acf7-702079789dcb-service-ca\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.663500 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.663411    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q72m9\" (UniqueName: \"kubernetes.io/projected/86e8b3d0-df94-4f49-acf7-702079789dcb-kube-api-access-q72m9\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.663500 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.663446    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86e8b3d0-df94-4f49-acf7-702079789dcb-console-serving-cert\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.664008 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.663977    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86e8b3d0-df94-4f49-acf7-702079789dcb-console-config\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.664136 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.664023    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86e8b3d0-df94-4f49-acf7-702079789dcb-oauth-serving-cert\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.664136 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.664070    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86e8b3d0-df94-4f49-acf7-702079789dcb-service-ca\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.664136 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.664098    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86e8b3d0-df94-4f49-acf7-702079789dcb-trusted-ca-bundle\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.665793 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.665775    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86e8b3d0-df94-4f49-acf7-702079789dcb-console-oauth-config\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.665900 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.665874    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86e8b3d0-df94-4f49-acf7-702079789dcb-console-serving-cert\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.675058 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.675033    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q72m9\" (UniqueName: \"kubernetes.io/projected/86e8b3d0-df94-4f49-acf7-702079789dcb-kube-api-access-q72m9\") pod \"console-5cc6bf9c98-s9zcq\" (UID: \"86e8b3d0-df94-4f49-acf7-702079789dcb\") " pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.757375 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.757343    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:28.890952 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.890920    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cc6bf9c98-s9zcq"]
May 25 13:13:28.894129 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:13:28.894099    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86e8b3d0_df94_4f49_acf7_702079789dcb.slice/crio-946fb5e647b61502981816fe55a1cf043f3e6dc2ff58ec674ac14ce816dafc43 WatchSource:0}: Error finding container 946fb5e647b61502981816fe55a1cf043f3e6dc2ff58ec674ac14ce816dafc43: Status 404 returned error can't find the container with id 946fb5e647b61502981816fe55a1cf043f3e6dc2ff58ec674ac14ce816dafc43
May 25 13:13:28.895904 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:28.895890    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 13:13:29.303442 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:29.303408    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cc6bf9c98-s9zcq" event={"ID":"86e8b3d0-df94-4f49-acf7-702079789dcb","Type":"ContainerStarted","Data":"73b64e8d3f2af2f21c8e0820794a3f32f30166dee34f379e3f0ef65591f5bc82"}
May 25 13:13:29.303442 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:29.303444    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cc6bf9c98-s9zcq" event={"ID":"86e8b3d0-df94-4f49-acf7-702079789dcb","Type":"ContainerStarted","Data":"946fb5e647b61502981816fe55a1cf043f3e6dc2ff58ec674ac14ce816dafc43"}
May 25 13:13:29.324204 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:29.324158    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5cc6bf9c98-s9zcq" podStartSLOduration=1.32414537 podStartE2EDuration="1.32414537s" podCreationTimestamp="2026-05-25 13:13:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:13:29.324115216 +0000 UTC m=+476.043078352" watchObservedRunningTime="2026-05-25 13:13:29.32414537 +0000 UTC m=+476.043108508"
May 25 13:13:38.758266 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:38.758185    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:38.758266 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:38.758224    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:38.762921 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:38.762894    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:39.334413 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:39.334385    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cc6bf9c98-s9zcq"
May 25 13:13:39.403386 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:13:39.403357    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f7447dd6b-vlp6b"]
May 25 13:14:04.423798 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.423713    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7f7447dd6b-vlp6b" podUID="d21b804b-2f92-4980-9c6b-bf60a0c583d0" containerName="console" containerID="cri-o://d8992215fc855cc7f7dd807832177b477a9524066b6637294ac4f0257aa20eb1" gracePeriod=15
May 25 13:14:04.573687 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.573646    2559 patch_prober.go:28] interesting pod/console-7f7447dd6b-vlp6b container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.133.0.20:8443/health\": dial tcp 10.133.0.20:8443: connect: connection refused" start-of-body=
May 25 13:14:04.573876 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.573743    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-7f7447dd6b-vlp6b" podUID="d21b804b-2f92-4980-9c6b-bf60a0c583d0" containerName="console" probeResult="failure" output="Get \"https://10.133.0.20:8443/health\": dial tcp 10.133.0.20:8443: connect: connection refused"
May 25 13:14:04.661483 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.661463    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7f7447dd6b-vlp6b_d21b804b-2f92-4980-9c6b-bf60a0c583d0/console/0.log"
May 25 13:14:04.661580 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.661522    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:14:04.751918 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.751882    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-oauth-config\") pod \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") "
May 25 13:14:04.751918 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.751928    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-oauth-serving-cert\") pod \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") "
May 25 13:14:04.752186 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.751956    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-trusted-ca-bundle\") pod \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") "
May 25 13:14:04.752186 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.751995    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-serving-cert\") pod \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") "
May 25 13:14:04.752186 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.752027    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q424k\" (UniqueName: \"kubernetes.io/projected/d21b804b-2f92-4980-9c6b-bf60a0c583d0-kube-api-access-q424k\") pod \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") "
May 25 13:14:04.752186 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.752081    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-service-ca\") pod \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") "
May 25 13:14:04.752186 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.752109    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-config\") pod \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\" (UID: \"d21b804b-2f92-4980-9c6b-bf60a0c583d0\") "
May 25 13:14:04.752458 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.752415    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d21b804b-2f92-4980-9c6b-bf60a0c583d0" (UID: "d21b804b-2f92-4980-9c6b-bf60a0c583d0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:14:04.752571 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.752546    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d21b804b-2f92-4980-9c6b-bf60a0c583d0" (UID: "d21b804b-2f92-4980-9c6b-bf60a0c583d0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:14:04.752618 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.752566    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-service-ca" (OuterVolumeSpecName: "service-ca") pod "d21b804b-2f92-4980-9c6b-bf60a0c583d0" (UID: "d21b804b-2f92-4980-9c6b-bf60a0c583d0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:14:04.752669 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.752647    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-config" (OuterVolumeSpecName: "console-config") pod "d21b804b-2f92-4980-9c6b-bf60a0c583d0" (UID: "d21b804b-2f92-4980-9c6b-bf60a0c583d0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:14:04.754366 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.754322    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21b804b-2f92-4980-9c6b-bf60a0c583d0-kube-api-access-q424k" (OuterVolumeSpecName: "kube-api-access-q424k") pod "d21b804b-2f92-4980-9c6b-bf60a0c583d0" (UID: "d21b804b-2f92-4980-9c6b-bf60a0c583d0"). InnerVolumeSpecName "kube-api-access-q424k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:14:04.754695 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.754670    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d21b804b-2f92-4980-9c6b-bf60a0c583d0" (UID: "d21b804b-2f92-4980-9c6b-bf60a0c583d0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:14:04.754794 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.754695    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d21b804b-2f92-4980-9c6b-bf60a0c583d0" (UID: "d21b804b-2f92-4980-9c6b-bf60a0c583d0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:14:04.853071 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.853031    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-service-ca\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:14:04.853071 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.853064    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-config\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:14:04.853071 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.853077    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-oauth-config\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:14:04.853295 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.853090    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-oauth-serving-cert\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:14:04.853295 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.853103    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d21b804b-2f92-4980-9c6b-bf60a0c583d0-trusted-ca-bundle\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:14:04.853295 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.853115    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d21b804b-2f92-4980-9c6b-bf60a0c583d0-console-serving-cert\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:14:04.853295 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:04.853127    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q424k\" (UniqueName: \"kubernetes.io/projected/d21b804b-2f92-4980-9c6b-bf60a0c583d0-kube-api-access-q424k\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:14:05.395050 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:05.395024    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7f7447dd6b-vlp6b_d21b804b-2f92-4980-9c6b-bf60a0c583d0/console/0.log"
May 25 13:14:05.395222 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:05.395060    2559 generic.go:358] "Generic (PLEG): container finished" podID="d21b804b-2f92-4980-9c6b-bf60a0c583d0" containerID="d8992215fc855cc7f7dd807832177b477a9524066b6637294ac4f0257aa20eb1" exitCode=2
May 25 13:14:05.395222 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:05.395094    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7447dd6b-vlp6b" event={"ID":"d21b804b-2f92-4980-9c6b-bf60a0c583d0","Type":"ContainerDied","Data":"d8992215fc855cc7f7dd807832177b477a9524066b6637294ac4f0257aa20eb1"}
May 25 13:14:05.395222 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:05.395116    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f7447dd6b-vlp6b" event={"ID":"d21b804b-2f92-4980-9c6b-bf60a0c583d0","Type":"ContainerDied","Data":"65455599b371fc462b3e14ec9d45f5c24421017fe49e68f39092f467de803730"}
May 25 13:14:05.395222 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:05.395127    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f7447dd6b-vlp6b"
May 25 13:14:05.395222 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:05.395130    2559 scope.go:117] "RemoveContainer" containerID="d8992215fc855cc7f7dd807832177b477a9524066b6637294ac4f0257aa20eb1"
May 25 13:14:05.402970 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:05.402939    2559 scope.go:117] "RemoveContainer" containerID="d8992215fc855cc7f7dd807832177b477a9524066b6637294ac4f0257aa20eb1"
May 25 13:14:05.403208 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:14:05.403190    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8992215fc855cc7f7dd807832177b477a9524066b6637294ac4f0257aa20eb1\": container with ID starting with d8992215fc855cc7f7dd807832177b477a9524066b6637294ac4f0257aa20eb1 not found: ID does not exist" containerID="d8992215fc855cc7f7dd807832177b477a9524066b6637294ac4f0257aa20eb1"
May 25 13:14:05.403272 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:05.403221    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8992215fc855cc7f7dd807832177b477a9524066b6637294ac4f0257aa20eb1"} err="failed to get container status \"d8992215fc855cc7f7dd807832177b477a9524066b6637294ac4f0257aa20eb1\": rpc error: code = NotFound desc = could not find container \"d8992215fc855cc7f7dd807832177b477a9524066b6637294ac4f0257aa20eb1\": container with ID starting with d8992215fc855cc7f7dd807832177b477a9524066b6637294ac4f0257aa20eb1 not found: ID does not exist"
May 25 13:14:05.415208 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:05.415187    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f7447dd6b-vlp6b"]
May 25 13:14:05.419099 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:05.419079    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f7447dd6b-vlp6b"]
May 25 13:14:05.868528 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:14:05.868493    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d21b804b-2f92-4980-9c6b-bf60a0c583d0" path="/var/lib/kubelet/pods/d21b804b-2f92-4980-9c6b-bf60a0c583d0/volumes"
May 25 13:18:32.209070 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.209034    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"]
May 25 13:18:32.209467 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.209281    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d21b804b-2f92-4980-9c6b-bf60a0c583d0" containerName="console"
May 25 13:18:32.209467 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.209290    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21b804b-2f92-4980-9c6b-bf60a0c583d0" containerName="console"
May 25 13:18:32.209467 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.209329    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="d21b804b-2f92-4980-9c6b-bf60a0c583d0" containerName="console"
May 25 13:18:32.212304 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.212289    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"
May 25 13:18:32.215386 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.215362    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bxf9p\""
May 25 13:18:32.215520 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.215385    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 25 13:18:32.216269 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.216251    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 25 13:18:32.237625 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.237601    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"]
May 25 13:18:32.303385 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.303344    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2953b7b0-b2e2-433c-ae39-1260266df34b-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms\" (UID: \"2953b7b0-b2e2-433c-ae39-1260266df34b\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"
May 25 13:18:32.303385 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.303390    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8kcc\" (UniqueName: \"kubernetes.io/projected/2953b7b0-b2e2-433c-ae39-1260266df34b-kube-api-access-f8kcc\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms\" (UID: \"2953b7b0-b2e2-433c-ae39-1260266df34b\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"
May 25 13:18:32.303585 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.303475    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2953b7b0-b2e2-433c-ae39-1260266df34b-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms\" (UID: \"2953b7b0-b2e2-433c-ae39-1260266df34b\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"
May 25 13:18:32.403879 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.403834    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2953b7b0-b2e2-433c-ae39-1260266df34b-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms\" (UID: \"2953b7b0-b2e2-433c-ae39-1260266df34b\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"
May 25 13:18:32.404063 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.403905    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2953b7b0-b2e2-433c-ae39-1260266df34b-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms\" (UID: \"2953b7b0-b2e2-433c-ae39-1260266df34b\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"
May 25 13:18:32.404063 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.403932    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f8kcc\" (UniqueName: \"kubernetes.io/projected/2953b7b0-b2e2-433c-ae39-1260266df34b-kube-api-access-f8kcc\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms\" (UID: \"2953b7b0-b2e2-433c-ae39-1260266df34b\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"
May 25 13:18:32.404226 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.404207    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2953b7b0-b2e2-433c-ae39-1260266df34b-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms\" (UID: \"2953b7b0-b2e2-433c-ae39-1260266df34b\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"
May 25 13:18:32.404294 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.404273    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2953b7b0-b2e2-433c-ae39-1260266df34b-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms\" (UID: \"2953b7b0-b2e2-433c-ae39-1260266df34b\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"
May 25 13:18:32.413229 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.413212    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8kcc\" (UniqueName: \"kubernetes.io/projected/2953b7b0-b2e2-433c-ae39-1260266df34b-kube-api-access-f8kcc\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms\" (UID: \"2953b7b0-b2e2-433c-ae39-1260266df34b\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"
May 25 13:18:32.520355 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.520319    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"
May 25 13:18:32.642371 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.642346    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"]
May 25 13:18:32.644589 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:18:32.644550    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2953b7b0_b2e2_433c_ae39_1260266df34b.slice/crio-e80684e928975703fd1c2234a10504dbdfce5d1f0cb4a68fad9d19618096fdef WatchSource:0}: Error finding container e80684e928975703fd1c2234a10504dbdfce5d1f0cb4a68fad9d19618096fdef: Status 404 returned error can't find the container with id e80684e928975703fd1c2234a10504dbdfce5d1f0cb4a68fad9d19618096fdef
May 25 13:18:32.646319 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:32.646304    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 13:18:33.088947 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:33.088912    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms" event={"ID":"2953b7b0-b2e2-433c-ae39-1260266df34b","Type":"ContainerStarted","Data":"e80684e928975703fd1c2234a10504dbdfce5d1f0cb4a68fad9d19618096fdef"}
May 25 13:18:39.107299 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:39.107263    2559 generic.go:358] "Generic (PLEG): container finished" podID="2953b7b0-b2e2-433c-ae39-1260266df34b" containerID="1437f84f83f46e9e3564c22e4b1000bb159ece4d4fadfaf25aff2669d66059a1" exitCode=0
May 25 13:18:39.107667 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:39.107317    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms" event={"ID":"2953b7b0-b2e2-433c-ae39-1260266df34b","Type":"ContainerDied","Data":"1437f84f83f46e9e3564c22e4b1000bb159ece4d4fadfaf25aff2669d66059a1"}
May 25 13:18:42.116385 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:42.116352    2559 generic.go:358] "Generic (PLEG): container finished" podID="2953b7b0-b2e2-433c-ae39-1260266df34b" containerID="87f79e7fead60794b81f0bb4514861fec4fe5c79ee947d248b5f6262181e6c08" exitCode=0
May 25 13:18:42.116761 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:42.116406    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms" event={"ID":"2953b7b0-b2e2-433c-ae39-1260266df34b","Type":"ContainerDied","Data":"87f79e7fead60794b81f0bb4514861fec4fe5c79ee947d248b5f6262181e6c08"}
May 25 13:18:49.137536 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:49.137452    2559 generic.go:358] "Generic (PLEG): container finished" podID="2953b7b0-b2e2-433c-ae39-1260266df34b" containerID="2e40fd943ef3e936683d6a0de744c7b6419f2762ccbd37da747b807d639b6da7" exitCode=0
May 25 13:18:49.137536 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:49.137526    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms" event={"ID":"2953b7b0-b2e2-433c-ae39-1260266df34b","Type":"ContainerDied","Data":"2e40fd943ef3e936683d6a0de744c7b6419f2762ccbd37da747b807d639b6da7"}
May 25 13:18:50.253731 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:50.253695    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"
May 25 13:18:50.351693 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:50.351666    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2953b7b0-b2e2-433c-ae39-1260266df34b-bundle\") pod \"2953b7b0-b2e2-433c-ae39-1260266df34b\" (UID: \"2953b7b0-b2e2-433c-ae39-1260266df34b\") "
May 25 13:18:50.351881 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:50.351766    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8kcc\" (UniqueName: \"kubernetes.io/projected/2953b7b0-b2e2-433c-ae39-1260266df34b-kube-api-access-f8kcc\") pod \"2953b7b0-b2e2-433c-ae39-1260266df34b\" (UID: \"2953b7b0-b2e2-433c-ae39-1260266df34b\") "
May 25 13:18:50.351881 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:50.351783    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2953b7b0-b2e2-433c-ae39-1260266df34b-util\") pod \"2953b7b0-b2e2-433c-ae39-1260266df34b\" (UID: \"2953b7b0-b2e2-433c-ae39-1260266df34b\") "
May 25 13:18:50.352227 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:50.352200    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2953b7b0-b2e2-433c-ae39-1260266df34b-bundle" (OuterVolumeSpecName: "bundle") pod "2953b7b0-b2e2-433c-ae39-1260266df34b" (UID: "2953b7b0-b2e2-433c-ae39-1260266df34b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:18:50.353899 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:50.353852    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2953b7b0-b2e2-433c-ae39-1260266df34b-kube-api-access-f8kcc" (OuterVolumeSpecName: "kube-api-access-f8kcc") pod "2953b7b0-b2e2-433c-ae39-1260266df34b" (UID: "2953b7b0-b2e2-433c-ae39-1260266df34b"). InnerVolumeSpecName "kube-api-access-f8kcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:18:50.356031 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:50.356012    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2953b7b0-b2e2-433c-ae39-1260266df34b-util" (OuterVolumeSpecName: "util") pod "2953b7b0-b2e2-433c-ae39-1260266df34b" (UID: "2953b7b0-b2e2-433c-ae39-1260266df34b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:18:50.452979 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:50.452906    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8kcc\" (UniqueName: \"kubernetes.io/projected/2953b7b0-b2e2-433c-ae39-1260266df34b-kube-api-access-f8kcc\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:18:50.452979 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:50.452934    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2953b7b0-b2e2-433c-ae39-1260266df34b-util\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:18:50.452979 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:50.452944    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2953b7b0-b2e2-433c-ae39-1260266df34b-bundle\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:18:51.145295 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:51.145255    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms" event={"ID":"2953b7b0-b2e2-433c-ae39-1260266df34b","Type":"ContainerDied","Data":"e80684e928975703fd1c2234a10504dbdfce5d1f0cb4a68fad9d19618096fdef"}
May 25 13:18:51.145295 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:51.145296    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e80684e928975703fd1c2234a10504dbdfce5d1f0cb4a68fad9d19618096fdef"
May 25 13:18:51.145493 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:51.145337    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cg46ms"
May 25 13:18:54.258490 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.258445    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt"]
May 25 13:18:54.258884 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.258805    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2953b7b0-b2e2-433c-ae39-1260266df34b" containerName="extract"
May 25 13:18:54.258884 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.258822    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2953b7b0-b2e2-433c-ae39-1260266df34b" containerName="extract"
May 25 13:18:54.258884 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.258848    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2953b7b0-b2e2-433c-ae39-1260266df34b" containerName="util"
May 25 13:18:54.258884 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.258856    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2953b7b0-b2e2-433c-ae39-1260266df34b" containerName="util"
May 25 13:18:54.258884 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.258870    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2953b7b0-b2e2-433c-ae39-1260266df34b" containerName="pull"
May 25 13:18:54.258884 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.258878    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2953b7b0-b2e2-433c-ae39-1260266df34b" containerName="pull"
May 25 13:18:54.259090 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.258935    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="2953b7b0-b2e2-433c-ae39-1260266df34b" containerName="extract"
May 25 13:18:54.310733 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.310698    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt"]
May 25 13:18:54.310873 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.310834    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt"
May 25 13:18:54.313188 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.313163    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
May 25 13:18:54.313188 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.313178    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
May 25 13:18:54.313188 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.313179    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-cxsw2\""
May 25 13:18:54.313380 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.313325    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
May 25 13:18:54.382231 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.382206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/eca91d21-b407-4394-ac8e-9ae96c6ba77d-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt\" (UID: \"eca91d21-b407-4394-ac8e-9ae96c6ba77d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt"
May 25 13:18:54.382393 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.382241    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68s9p\" (UniqueName: \"kubernetes.io/projected/eca91d21-b407-4394-ac8e-9ae96c6ba77d-kube-api-access-68s9p\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt\" (UID: \"eca91d21-b407-4394-ac8e-9ae96c6ba77d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt"
May 25 13:18:54.482626 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.482596    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/eca91d21-b407-4394-ac8e-9ae96c6ba77d-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt\" (UID: \"eca91d21-b407-4394-ac8e-9ae96c6ba77d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt"
May 25 13:18:54.482789 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.482632    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-68s9p\" (UniqueName: \"kubernetes.io/projected/eca91d21-b407-4394-ac8e-9ae96c6ba77d-kube-api-access-68s9p\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt\" (UID: \"eca91d21-b407-4394-ac8e-9ae96c6ba77d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt"
May 25 13:18:54.484928 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.484902    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/eca91d21-b407-4394-ac8e-9ae96c6ba77d-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt\" (UID: \"eca91d21-b407-4394-ac8e-9ae96c6ba77d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt"
May 25 13:18:54.490789 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.490767    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68s9p\" (UniqueName: \"kubernetes.io/projected/eca91d21-b407-4394-ac8e-9ae96c6ba77d-kube-api-access-68s9p\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt\" (UID: \"eca91d21-b407-4394-ac8e-9ae96c6ba77d\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt"
May 25 13:18:54.620636 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.620559    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt"
May 25 13:18:54.745522 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:54.745496    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt"]
May 25 13:18:54.747521 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:18:54.747491    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca91d21_b407_4394_ac8e_9ae96c6ba77d.slice/crio-0f254f8805656a12f167b121e432a33dc9be454356035da78e5cfbdb94c06fb2 WatchSource:0}: Error finding container 0f254f8805656a12f167b121e432a33dc9be454356035da78e5cfbdb94c06fb2: Status 404 returned error can't find the container with id 0f254f8805656a12f167b121e432a33dc9be454356035da78e5cfbdb94c06fb2
May 25 13:18:55.157707 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:55.157665    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt" event={"ID":"eca91d21-b407-4394-ac8e-9ae96c6ba77d","Type":"ContainerStarted","Data":"0f254f8805656a12f167b121e432a33dc9be454356035da78e5cfbdb94c06fb2"}
May 25 13:18:58.938187 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:58.938152    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-tsdlt"]
May 25 13:18:58.955524 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:58.955494    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-tsdlt"]
May 25 13:18:58.955678 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:58.955624    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:18:58.958088 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:58.958063    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
May 25 13:18:58.959015 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:58.958993    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
May 25 13:18:58.959548 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:58.959532    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-87cm2\""
May 25 13:18:59.125230 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.125189    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates\") pod \"keda-operator-ffbb595cb-tsdlt\" (UID: \"39e3a9a5-d108-4e49-8597-1b2809fbbb49\") " pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:18:59.125230 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.125234    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/39e3a9a5-d108-4e49-8597-1b2809fbbb49-cabundle0\") pod \"keda-operator-ffbb595cb-tsdlt\" (UID: \"39e3a9a5-d108-4e49-8597-1b2809fbbb49\") " pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:18:59.125467 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.125334    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdxln\" (UniqueName: \"kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-kube-api-access-pdxln\") pod \"keda-operator-ffbb595cb-tsdlt\" (UID: \"39e3a9a5-d108-4e49-8597-1b2809fbbb49\") " pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:18:59.172547 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.172509    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt" event={"ID":"eca91d21-b407-4394-ac8e-9ae96c6ba77d","Type":"ContainerStarted","Data":"23dcb201e8d6af1707d83967518634ea01e98363aa616dc01dbf672590016c4c"}
May 25 13:18:59.172734 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.172647    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt"
May 25 13:18:59.226527 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.226447    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates\") pod \"keda-operator-ffbb595cb-tsdlt\" (UID: \"39e3a9a5-d108-4e49-8597-1b2809fbbb49\") " pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:18:59.226527 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.226505    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/39e3a9a5-d108-4e49-8597-1b2809fbbb49-cabundle0\") pod \"keda-operator-ffbb595cb-tsdlt\" (UID: \"39e3a9a5-d108-4e49-8597-1b2809fbbb49\") " pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:18:59.226768 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.226559    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pdxln\" (UniqueName: \"kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-kube-api-access-pdxln\") pod \"keda-operator-ffbb595cb-tsdlt\" (UID: \"39e3a9a5-d108-4e49-8597-1b2809fbbb49\") " pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:18:59.226768 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:18:59.226646    2559 secret.go:281] references non-existent secret key: ca.crt
May 25 13:18:59.226768 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:18:59.226669    2559 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
May 25 13:18:59.226768 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:18:59.226681    2559 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-tsdlt: references non-existent secret key: ca.crt
May 25 13:18:59.226768 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:18:59.226748    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates podName:39e3a9a5-d108-4e49-8597-1b2809fbbb49 nodeName:}" failed. No retries permitted until 2026-05-25 13:18:59.726714603 +0000 UTC m=+806.445677720 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates") pod "keda-operator-ffbb595cb-tsdlt" (UID: "39e3a9a5-d108-4e49-8597-1b2809fbbb49") : references non-existent secret key: ca.crt
May 25 13:18:59.227305 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.227274    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/39e3a9a5-d108-4e49-8597-1b2809fbbb49-cabundle0\") pod \"keda-operator-ffbb595cb-tsdlt\" (UID: \"39e3a9a5-d108-4e49-8597-1b2809fbbb49\") " pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:18:59.228311 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.228256    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt" podStartSLOduration=1.623367189 podStartE2EDuration="5.228241303s" podCreationTimestamp="2026-05-25 13:18:54 +0000 UTC" firstStartedPulling="2026-05-25 13:18:54.749189313 +0000 UTC m=+801.468152429" lastFinishedPulling="2026-05-25 13:18:58.354063423 +0000 UTC m=+805.073026543" observedRunningTime="2026-05-25 13:18:59.22692444 +0000 UTC m=+805.945887580" watchObservedRunningTime="2026-05-25 13:18:59.228241303 +0000 UTC m=+805.947204443"
May 25 13:18:59.248503 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.248475    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdxln\" (UniqueName: \"kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-kube-api-access-pdxln\") pod \"keda-operator-ffbb595cb-tsdlt\" (UID: \"39e3a9a5-d108-4e49-8597-1b2809fbbb49\") " pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:18:59.381812 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.381777    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"]
May 25 13:18:59.383991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.383974    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:18:59.386145 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.386125    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
May 25 13:18:59.396971 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.396951    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"]
May 25 13:18:59.529600 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.529569    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-certificates\") pod \"keda-metrics-apiserver-7c9f485588-5hjjv\" (UID: \"121e03c2-69ea-475b-a9cd-3e321ff19c0c\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:18:59.529784 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.529650    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4v9z\" (UniqueName: \"kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-kube-api-access-w4v9z\") pod \"keda-metrics-apiserver-7c9f485588-5hjjv\" (UID: \"121e03c2-69ea-475b-a9cd-3e321ff19c0c\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:18:59.529784 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.529687    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/121e03c2-69ea-475b-a9cd-3e321ff19c0c-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-5hjjv\" (UID: \"121e03c2-69ea-475b-a9cd-3e321ff19c0c\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:18:59.630738 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.630680    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-certificates\") pod \"keda-metrics-apiserver-7c9f485588-5hjjv\" (UID: \"121e03c2-69ea-475b-a9cd-3e321ff19c0c\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:18:59.630926 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.630777    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w4v9z\" (UniqueName: \"kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-kube-api-access-w4v9z\") pod \"keda-metrics-apiserver-7c9f485588-5hjjv\" (UID: \"121e03c2-69ea-475b-a9cd-3e321ff19c0c\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:18:59.630926 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.630818    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/121e03c2-69ea-475b-a9cd-3e321ff19c0c-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-5hjjv\" (UID: \"121e03c2-69ea-475b-a9cd-3e321ff19c0c\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:18:59.630926 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:18:59.630866    2559 secret.go:281] references non-existent secret key: tls.crt
May 25 13:18:59.630926 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:18:59.630891    2559 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
May 25 13:18:59.630926 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:18:59.630909    2559 projected.go:264] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found
May 25 13:18:59.631170 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:18:59.630931    2559 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv: [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found]
May 25 13:18:59.631170 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:18:59.630997    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-certificates podName:121e03c2-69ea-475b-a9cd-3e321ff19c0c nodeName:}" failed. No retries permitted until 2026-05-25 13:19:00.130978469 +0000 UTC m=+806.849941601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-certificates") pod "keda-metrics-apiserver-7c9f485588-5hjjv" (UID: "121e03c2-69ea-475b-a9cd-3e321ff19c0c") : [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found]
May 25 13:18:59.631258 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.631212    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/121e03c2-69ea-475b-a9cd-3e321ff19c0c-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-5hjjv\" (UID: \"121e03c2-69ea-475b-a9cd-3e321ff19c0c\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:18:59.665034 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.664997    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4v9z\" (UniqueName: \"kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-kube-api-access-w4v9z\") pod \"keda-metrics-apiserver-7c9f485588-5hjjv\" (UID: \"121e03c2-69ea-475b-a9cd-3e321ff19c0c\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:18:59.732024 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.731989    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates\") pod \"keda-operator-ffbb595cb-tsdlt\" (UID: \"39e3a9a5-d108-4e49-8597-1b2809fbbb49\") " pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:18:59.732183 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:18:59.732104    2559 secret.go:281] references non-existent secret key: ca.crt
May 25 13:18:59.732183 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:18:59.732116    2559 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
May 25 13:18:59.732183 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:18:59.732126    2559 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-tsdlt: references non-existent secret key: ca.crt
May 25 13:18:59.732183 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:18:59.732173    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates podName:39e3a9a5-d108-4e49-8597-1b2809fbbb49 nodeName:}" failed. No retries permitted until 2026-05-25 13:19:00.732159186 +0000 UTC m=+807.451122303 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates") pod "keda-operator-ffbb595cb-tsdlt" (UID: "39e3a9a5-d108-4e49-8597-1b2809fbbb49") : references non-existent secret key: ca.crt
May 25 13:18:59.874908 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.874836    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-cf49989db-zkzv7"]
May 25 13:18:59.876879 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.876864    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-cf49989db-zkzv7"
May 25 13:18:59.882138 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.882114    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
May 25 13:18:59.892768 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:18:59.892745    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-cf49989db-zkzv7"]
May 25 13:19:00.034033 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:00.034003    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1c348bb7-54e7-4118-b57c-bf8a2928e1f2-certificates\") pod \"keda-admission-cf49989db-zkzv7\" (UID: \"1c348bb7-54e7-4118-b57c-bf8a2928e1f2\") " pod="openshift-keda/keda-admission-cf49989db-zkzv7"
May 25 13:19:00.034385 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:00.034051    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n99xw\" (UniqueName: \"kubernetes.io/projected/1c348bb7-54e7-4118-b57c-bf8a2928e1f2-kube-api-access-n99xw\") pod \"keda-admission-cf49989db-zkzv7\" (UID: \"1c348bb7-54e7-4118-b57c-bf8a2928e1f2\") " pod="openshift-keda/keda-admission-cf49989db-zkzv7"
May 25 13:19:00.134535 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:00.134458    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1c348bb7-54e7-4118-b57c-bf8a2928e1f2-certificates\") pod \"keda-admission-cf49989db-zkzv7\" (UID: \"1c348bb7-54e7-4118-b57c-bf8a2928e1f2\") " pod="openshift-keda/keda-admission-cf49989db-zkzv7"
May 25 13:19:00.134535 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:00.134497    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n99xw\" (UniqueName: \"kubernetes.io/projected/1c348bb7-54e7-4118-b57c-bf8a2928e1f2-kube-api-access-n99xw\") pod \"keda-admission-cf49989db-zkzv7\" (UID: \"1c348bb7-54e7-4118-b57c-bf8a2928e1f2\") " pod="openshift-keda/keda-admission-cf49989db-zkzv7"
May 25 13:19:00.134735 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:00.134564    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-certificates\") pod \"keda-metrics-apiserver-7c9f485588-5hjjv\" (UID: \"121e03c2-69ea-475b-a9cd-3e321ff19c0c\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:19:00.134735 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:00.134641    2559 secret.go:281] references non-existent secret key: tls.crt
May 25 13:19:00.134735 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:00.134653    2559 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
May 25 13:19:00.134735 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:00.134669    2559 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv: references non-existent secret key: tls.crt
May 25 13:19:00.134735 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:00.134712    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-certificates podName:121e03c2-69ea-475b-a9cd-3e321ff19c0c nodeName:}" failed. No retries permitted until 2026-05-25 13:19:01.134698983 +0000 UTC m=+807.853662100 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-certificates") pod "keda-metrics-apiserver-7c9f485588-5hjjv" (UID: "121e03c2-69ea-475b-a9cd-3e321ff19c0c") : references non-existent secret key: tls.crt
May 25 13:19:00.136933 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:00.136909    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1c348bb7-54e7-4118-b57c-bf8a2928e1f2-certificates\") pod \"keda-admission-cf49989db-zkzv7\" (UID: \"1c348bb7-54e7-4118-b57c-bf8a2928e1f2\") " pod="openshift-keda/keda-admission-cf49989db-zkzv7"
May 25 13:19:00.151713 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:00.151681    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n99xw\" (UniqueName: \"kubernetes.io/projected/1c348bb7-54e7-4118-b57c-bf8a2928e1f2-kube-api-access-n99xw\") pod \"keda-admission-cf49989db-zkzv7\" (UID: \"1c348bb7-54e7-4118-b57c-bf8a2928e1f2\") " pod="openshift-keda/keda-admission-cf49989db-zkzv7"
May 25 13:19:00.187290 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:00.187263    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-cf49989db-zkzv7"
May 25 13:19:00.331857 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:00.331832    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-cf49989db-zkzv7"]
May 25 13:19:00.334081 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:19:00.334049    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c348bb7_54e7_4118_b57c_bf8a2928e1f2.slice/crio-09044a14ddb510099fca654499858d32b059a5630d0d21b6730a710ff555479c WatchSource:0}: Error finding container 09044a14ddb510099fca654499858d32b059a5630d0d21b6730a710ff555479c: Status 404 returned error can't find the container with id 09044a14ddb510099fca654499858d32b059a5630d0d21b6730a710ff555479c
May 25 13:19:00.738681 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:00.738644    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates\") pod \"keda-operator-ffbb595cb-tsdlt\" (UID: \"39e3a9a5-d108-4e49-8597-1b2809fbbb49\") " pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:19:00.738865 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:00.738810    2559 secret.go:281] references non-existent secret key: ca.crt
May 25 13:19:00.738865 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:00.738833    2559 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
May 25 13:19:00.738865 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:00.738843    2559 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-tsdlt: references non-existent secret key: ca.crt
May 25 13:19:00.738968 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:00.738902    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates podName:39e3a9a5-d108-4e49-8597-1b2809fbbb49 nodeName:}" failed. No retries permitted until 2026-05-25 13:19:02.738882426 +0000 UTC m=+809.457845546 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates") pod "keda-operator-ffbb595cb-tsdlt" (UID: "39e3a9a5-d108-4e49-8597-1b2809fbbb49") : references non-existent secret key: ca.crt
May 25 13:19:01.141700 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:01.141668    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-certificates\") pod \"keda-metrics-apiserver-7c9f485588-5hjjv\" (UID: \"121e03c2-69ea-475b-a9cd-3e321ff19c0c\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:19:01.142091 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:01.141822    2559 secret.go:281] references non-existent secret key: tls.crt
May 25 13:19:01.142091 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:01.141851    2559 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
May 25 13:19:01.142091 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:01.141871    2559 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv: references non-existent secret key: tls.crt
May 25 13:19:01.142091 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:01.141925    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-certificates podName:121e03c2-69ea-475b-a9cd-3e321ff19c0c nodeName:}" failed. No retries permitted until 2026-05-25 13:19:03.141911138 +0000 UTC m=+809.860874256 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-certificates") pod "keda-metrics-apiserver-7c9f485588-5hjjv" (UID: "121e03c2-69ea-475b-a9cd-3e321ff19c0c") : references non-existent secret key: tls.crt
May 25 13:19:01.179575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:01.179541    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-cf49989db-zkzv7" event={"ID":"1c348bb7-54e7-4118-b57c-bf8a2928e1f2","Type":"ContainerStarted","Data":"09044a14ddb510099fca654499858d32b059a5630d0d21b6730a710ff555479c"}
May 25 13:19:02.184277 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:02.184183    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-cf49989db-zkzv7" event={"ID":"1c348bb7-54e7-4118-b57c-bf8a2928e1f2","Type":"ContainerStarted","Data":"2d468e7c78d5063c09e0aad968a157697009c60e4ae840ddd9b92862af461f83"}
May 25 13:19:02.184689 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:02.184296    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-cf49989db-zkzv7"
May 25 13:19:02.226819 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:02.226772    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-admission-cf49989db-zkzv7" podStartSLOduration=1.741350552 podStartE2EDuration="3.226757579s" podCreationTimestamp="2026-05-25 13:18:59 +0000 UTC" firstStartedPulling="2026-05-25 13:19:00.335366133 +0000 UTC m=+807.054329251" lastFinishedPulling="2026-05-25 13:19:01.820773147 +0000 UTC m=+808.539736278" observedRunningTime="2026-05-25 13:19:02.2258762 +0000 UTC m=+808.944839322" watchObservedRunningTime="2026-05-25 13:19:02.226757579 +0000 UTC m=+808.945720743"
May 25 13:19:02.755192 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:02.755160    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates\") pod \"keda-operator-ffbb595cb-tsdlt\" (UID: \"39e3a9a5-d108-4e49-8597-1b2809fbbb49\") " pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:19:02.755354 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:02.755295    2559 secret.go:281] references non-existent secret key: ca.crt
May 25 13:19:02.755354 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:02.755308    2559 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
May 25 13:19:02.755354 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:02.755316    2559 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-tsdlt: references non-existent secret key: ca.crt
May 25 13:19:02.755471 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:19:02.755360    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates podName:39e3a9a5-d108-4e49-8597-1b2809fbbb49 nodeName:}" failed. No retries permitted until 2026-05-25 13:19:06.755348887 +0000 UTC m=+813.474312005 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates") pod "keda-operator-ffbb595cb-tsdlt" (UID: "39e3a9a5-d108-4e49-8597-1b2809fbbb49") : references non-existent secret key: ca.crt
May 25 13:19:03.158896 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:03.158792    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-certificates\") pod \"keda-metrics-apiserver-7c9f485588-5hjjv\" (UID: \"121e03c2-69ea-475b-a9cd-3e321ff19c0c\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:19:03.161451 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:03.161416    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/121e03c2-69ea-475b-a9cd-3e321ff19c0c-certificates\") pod \"keda-metrics-apiserver-7c9f485588-5hjjv\" (UID: \"121e03c2-69ea-475b-a9cd-3e321ff19c0c\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:19:03.296905 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:03.296860    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:19:03.429292 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:03.429269    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"]
May 25 13:19:03.431147 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:19:03.431124    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121e03c2_69ea_475b_a9cd_3e321ff19c0c.slice/crio-61c216ed90c3211bff2e122b013702dd7e5fd0a452a281461d6b31790efcc231 WatchSource:0}: Error finding container 61c216ed90c3211bff2e122b013702dd7e5fd0a452a281461d6b31790efcc231: Status 404 returned error can't find the container with id 61c216ed90c3211bff2e122b013702dd7e5fd0a452a281461d6b31790efcc231
May 25 13:19:04.190605 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:04.190571    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv" event={"ID":"121e03c2-69ea-475b-a9cd-3e321ff19c0c","Type":"ContainerStarted","Data":"61c216ed90c3211bff2e122b013702dd7e5fd0a452a281461d6b31790efcc231"}
May 25 13:19:06.791301 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:06.791261    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates\") pod \"keda-operator-ffbb595cb-tsdlt\" (UID: \"39e3a9a5-d108-4e49-8597-1b2809fbbb49\") " pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:19:06.793674 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:06.793653    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/39e3a9a5-d108-4e49-8597-1b2809fbbb49-certificates\") pod \"keda-operator-ffbb595cb-tsdlt\" (UID: \"39e3a9a5-d108-4e49-8597-1b2809fbbb49\") " pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:19:07.065451 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:07.065355    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:19:07.192914 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:07.192886    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-tsdlt"]
May 25 13:19:07.194791 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:19:07.194765    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39e3a9a5_d108_4e49_8597_1b2809fbbb49.slice/crio-b850dcb2ea7e7e0e87e25a1ceca5e45fa26e2a38e17f6496a1f90ebc9396a1cd WatchSource:0}: Error finding container b850dcb2ea7e7e0e87e25a1ceca5e45fa26e2a38e17f6496a1f90ebc9396a1cd: Status 404 returned error can't find the container with id b850dcb2ea7e7e0e87e25a1ceca5e45fa26e2a38e17f6496a1f90ebc9396a1cd
May 25 13:19:07.203172 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:07.203140    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv" event={"ID":"121e03c2-69ea-475b-a9cd-3e321ff19c0c","Type":"ContainerStarted","Data":"5b437b05be57c4316fee46b815f077379bbf51e7f0c739112564f8882e6be22c"}
May 25 13:19:07.203281 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:07.203247    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:19:07.204078 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:07.204059    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-tsdlt" event={"ID":"39e3a9a5-d108-4e49-8597-1b2809fbbb49","Type":"ContainerStarted","Data":"b850dcb2ea7e7e0e87e25a1ceca5e45fa26e2a38e17f6496a1f90ebc9396a1cd"}
May 25 13:19:07.224466 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:07.224423    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv" podStartSLOduration=5.521511021 podStartE2EDuration="8.224410278s" podCreationTimestamp="2026-05-25 13:18:59 +0000 UTC" firstStartedPulling="2026-05-25 13:19:03.432584649 +0000 UTC m=+810.151547771" lastFinishedPulling="2026-05-25 13:19:06.135483893 +0000 UTC m=+812.854447028" observedRunningTime="2026-05-25 13:19:07.223261509 +0000 UTC m=+813.942224648" watchObservedRunningTime="2026-05-25 13:19:07.224410278 +0000 UTC m=+813.943373416"
May 25 13:19:11.216859 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:11.216819    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-tsdlt" event={"ID":"39e3a9a5-d108-4e49-8597-1b2809fbbb49","Type":"ContainerStarted","Data":"46397cd2f532c326d888d9dbf41ab080111a078dffc756dd107651d0e20fc8a2"}
May 25 13:19:11.217222 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:11.216925    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:19:11.234835 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:11.234780    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-operator-ffbb595cb-tsdlt" podStartSLOduration=10.146169874 podStartE2EDuration="13.234764157s" podCreationTimestamp="2026-05-25 13:18:58 +0000 UTC" firstStartedPulling="2026-05-25 13:19:07.196251083 +0000 UTC m=+813.915214204" lastFinishedPulling="2026-05-25 13:19:10.284845357 +0000 UTC m=+817.003808487" observedRunningTime="2026-05-25 13:19:11.233380733 +0000 UTC m=+817.952343877" watchObservedRunningTime="2026-05-25 13:19:11.234764157 +0000 UTC m=+817.953727299"
May 25 13:19:18.211809 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:18.211773    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-5hjjv"
May 25 13:19:20.177554 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:20.177519    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-mmzmt"
May 25 13:19:23.189002 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:23.188963    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-cf49989db-zkzv7"
May 25 13:19:32.222290 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:32.222259    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-ffbb595cb-tsdlt"
May 25 13:19:52.614183 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.614144    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"]
May 25 13:19:52.616612 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.616593    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"
May 25 13:19:52.618886 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.618863    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 25 13:19:52.619002 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.618955    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bxf9p\""
May 25 13:19:52.619733 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.619703    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 25 13:19:52.626997 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.626976    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"]
May 25 13:19:52.733799 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.733755    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbfe0791-f4ac-4b71-84bb-2273278a02a7-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl\" (UID: \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"
May 25 13:19:52.733988 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.733832    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbfe0791-f4ac-4b71-84bb-2273278a02a7-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl\" (UID: \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"
May 25 13:19:52.733988 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.733881    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zscc\" (UniqueName: \"kubernetes.io/projected/fbfe0791-f4ac-4b71-84bb-2273278a02a7-kube-api-access-9zscc\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl\" (UID: \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"
May 25 13:19:52.834787 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.834741    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbfe0791-f4ac-4b71-84bb-2273278a02a7-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl\" (UID: \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"
May 25 13:19:52.834787 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.834796    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9zscc\" (UniqueName: \"kubernetes.io/projected/fbfe0791-f4ac-4b71-84bb-2273278a02a7-kube-api-access-9zscc\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl\" (UID: \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"
May 25 13:19:52.834989 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.834838    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbfe0791-f4ac-4b71-84bb-2273278a02a7-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl\" (UID: \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"
May 25 13:19:52.835179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.835158    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbfe0791-f4ac-4b71-84bb-2273278a02a7-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl\" (UID: \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"
May 25 13:19:52.835242 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.835169    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbfe0791-f4ac-4b71-84bb-2273278a02a7-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl\" (UID: \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"
May 25 13:19:52.846128 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.846102    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zscc\" (UniqueName: \"kubernetes.io/projected/fbfe0791-f4ac-4b71-84bb-2273278a02a7-kube-api-access-9zscc\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl\" (UID: \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"
May 25 13:19:52.925907 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:52.925825    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"
May 25 13:19:53.050120 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:53.050003    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"]
May 25 13:19:53.052756 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:19:53.052713    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbfe0791_f4ac_4b71_84bb_2273278a02a7.slice/crio-b6be59521211bfddaf3fb5ce099d95188bd515028b5d3477624514a7c5acb3cb WatchSource:0}: Error finding container b6be59521211bfddaf3fb5ce099d95188bd515028b5d3477624514a7c5acb3cb: Status 404 returned error can't find the container with id b6be59521211bfddaf3fb5ce099d95188bd515028b5d3477624514a7c5acb3cb
May 25 13:19:53.341604 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:53.341570    2559 generic.go:358] "Generic (PLEG): container finished" podID="fbfe0791-f4ac-4b71-84bb-2273278a02a7" containerID="cb36cc7633eb936bc30cb34f012e7749d7cd6323a55930566dec4af6e1a6166b" exitCode=0
May 25 13:19:53.341793 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:53.341658    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl" event={"ID":"fbfe0791-f4ac-4b71-84bb-2273278a02a7","Type":"ContainerDied","Data":"cb36cc7633eb936bc30cb34f012e7749d7cd6323a55930566dec4af6e1a6166b"}
May 25 13:19:53.341793 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:53.341695    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl" event={"ID":"fbfe0791-f4ac-4b71-84bb-2273278a02a7","Type":"ContainerStarted","Data":"b6be59521211bfddaf3fb5ce099d95188bd515028b5d3477624514a7c5acb3cb"}
May 25 13:19:55.350581 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:55.350545    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl" event={"ID":"fbfe0791-f4ac-4b71-84bb-2273278a02a7","Type":"ContainerStarted","Data":"397defdd4fcf4a163ccbeac79fa3007f9b11fc0e2e3b604f3e691e8c847a1839"}
May 25 13:19:56.356134 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:56.356101    2559 generic.go:358] "Generic (PLEG): container finished" podID="fbfe0791-f4ac-4b71-84bb-2273278a02a7" containerID="397defdd4fcf4a163ccbeac79fa3007f9b11fc0e2e3b604f3e691e8c847a1839" exitCode=0
May 25 13:19:56.356566 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:56.356141    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl" event={"ID":"fbfe0791-f4ac-4b71-84bb-2273278a02a7","Type":"ContainerDied","Data":"397defdd4fcf4a163ccbeac79fa3007f9b11fc0e2e3b604f3e691e8c847a1839"}
May 25 13:19:57.360945 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:57.360907    2559 generic.go:358] "Generic (PLEG): container finished" podID="fbfe0791-f4ac-4b71-84bb-2273278a02a7" containerID="f730b1af78d32d3fee13696368c3cc03bfa3ea23ae5d2c2fcf097a060ebb92ea" exitCode=0
May 25 13:19:57.361479 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:57.360995    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl" event={"ID":"fbfe0791-f4ac-4b71-84bb-2273278a02a7","Type":"ContainerDied","Data":"f730b1af78d32d3fee13696368c3cc03bfa3ea23ae5d2c2fcf097a060ebb92ea"}
May 25 13:19:58.482310 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:58.482283    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"
May 25 13:19:58.578137 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:58.578101    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zscc\" (UniqueName: \"kubernetes.io/projected/fbfe0791-f4ac-4b71-84bb-2273278a02a7-kube-api-access-9zscc\") pod \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\" (UID: \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\") "
May 25 13:19:58.578327 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:58.578158    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbfe0791-f4ac-4b71-84bb-2273278a02a7-util\") pod \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\" (UID: \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\") "
May 25 13:19:58.578327 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:58.578183    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbfe0791-f4ac-4b71-84bb-2273278a02a7-bundle\") pod \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\" (UID: \"fbfe0791-f4ac-4b71-84bb-2273278a02a7\") "
May 25 13:19:58.578874 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:58.578846    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbfe0791-f4ac-4b71-84bb-2273278a02a7-bundle" (OuterVolumeSpecName: "bundle") pod "fbfe0791-f4ac-4b71-84bb-2273278a02a7" (UID: "fbfe0791-f4ac-4b71-84bb-2273278a02a7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:19:58.580259 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:58.580233    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbfe0791-f4ac-4b71-84bb-2273278a02a7-kube-api-access-9zscc" (OuterVolumeSpecName: "kube-api-access-9zscc") pod "fbfe0791-f4ac-4b71-84bb-2273278a02a7" (UID: "fbfe0791-f4ac-4b71-84bb-2273278a02a7"). InnerVolumeSpecName "kube-api-access-9zscc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:19:58.667329 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:58.667249    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbfe0791-f4ac-4b71-84bb-2273278a02a7-util" (OuterVolumeSpecName: "util") pod "fbfe0791-f4ac-4b71-84bb-2273278a02a7" (UID: "fbfe0791-f4ac-4b71-84bb-2273278a02a7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:19:58.679061 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:58.679034    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zscc\" (UniqueName: \"kubernetes.io/projected/fbfe0791-f4ac-4b71-84bb-2273278a02a7-kube-api-access-9zscc\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:19:58.679061 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:58.679061    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbfe0791-f4ac-4b71-84bb-2273278a02a7-util\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:19:58.679198 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:58.679072    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbfe0791-f4ac-4b71-84bb-2273278a02a7-bundle\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:19:59.368379 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:59.368340    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl" event={"ID":"fbfe0791-f4ac-4b71-84bb-2273278a02a7","Type":"ContainerDied","Data":"b6be59521211bfddaf3fb5ce099d95188bd515028b5d3477624514a7c5acb3cb"}
May 25 13:19:59.368379 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:59.368380    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6be59521211bfddaf3fb5ce099d95188bd515028b5d3477624514a7c5acb3cb"
May 25 13:19:59.368578 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:19:59.368407    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dgkhzl"
May 25 13:20:14.620145 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.620064    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"]
May 25 13:20:14.620529 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.620367    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbfe0791-f4ac-4b71-84bb-2273278a02a7" containerName="util"
May 25 13:20:14.620529 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.620378    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbfe0791-f4ac-4b71-84bb-2273278a02a7" containerName="util"
May 25 13:20:14.620529 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.620391    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbfe0791-f4ac-4b71-84bb-2273278a02a7" containerName="pull"
May 25 13:20:14.620529 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.620397    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbfe0791-f4ac-4b71-84bb-2273278a02a7" containerName="pull"
May 25 13:20:14.620529 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.620404    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbfe0791-f4ac-4b71-84bb-2273278a02a7" containerName="extract"
May 25 13:20:14.620529 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.620409    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbfe0791-f4ac-4b71-84bb-2273278a02a7" containerName="extract"
May 25 13:20:14.620529 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.620458    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="fbfe0791-f4ac-4b71-84bb-2273278a02a7" containerName="extract"
May 25 13:20:14.625409 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.625389    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"
May 25 13:20:14.629834 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.629804    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bxf9p\""
May 25 13:20:14.629834 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.629830    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 25 13:20:14.630180 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.630167    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 25 13:20:14.644356 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.644331    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"]
May 25 13:20:14.709164 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.709124    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c387957-a263-450b-b5df-e541c32f1c9d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2\" (UID: \"3c387957-a263-450b-b5df-e541c32f1c9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"
May 25 13:20:14.709316 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.709192    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbdpd\" (UniqueName: \"kubernetes.io/projected/3c387957-a263-450b-b5df-e541c32f1c9d-kube-api-access-zbdpd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2\" (UID: \"3c387957-a263-450b-b5df-e541c32f1c9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"
May 25 13:20:14.709316 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.709251    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c387957-a263-450b-b5df-e541c32f1c9d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2\" (UID: \"3c387957-a263-450b-b5df-e541c32f1c9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"
May 25 13:20:14.810113 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.810069    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c387957-a263-450b-b5df-e541c32f1c9d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2\" (UID: \"3c387957-a263-450b-b5df-e541c32f1c9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"
May 25 13:20:14.810291 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.810134    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zbdpd\" (UniqueName: \"kubernetes.io/projected/3c387957-a263-450b-b5df-e541c32f1c9d-kube-api-access-zbdpd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2\" (UID: \"3c387957-a263-450b-b5df-e541c32f1c9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"
May 25 13:20:14.810291 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.810171    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c387957-a263-450b-b5df-e541c32f1c9d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2\" (UID: \"3c387957-a263-450b-b5df-e541c32f1c9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"
May 25 13:20:14.810554 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.810530    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c387957-a263-450b-b5df-e541c32f1c9d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2\" (UID: \"3c387957-a263-450b-b5df-e541c32f1c9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"
May 25 13:20:14.810595 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.810539    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c387957-a263-450b-b5df-e541c32f1c9d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2\" (UID: \"3c387957-a263-450b-b5df-e541c32f1c9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"
May 25 13:20:14.818657 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.818625    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbdpd\" (UniqueName: \"kubernetes.io/projected/3c387957-a263-450b-b5df-e541c32f1c9d-kube-api-access-zbdpd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2\" (UID: \"3c387957-a263-450b-b5df-e541c32f1c9d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"
May 25 13:20:14.951041 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:14.950949    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"
May 25 13:20:15.078945 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:15.078905    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"]
May 25 13:20:15.082562 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:20:15.082526    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c387957_a263_450b_b5df_e541c32f1c9d.slice/crio-6720e815f949672c4f99ccdd86aa2625153b0a9528e41bdaa8c5a0edf9f0c003 WatchSource:0}: Error finding container 6720e815f949672c4f99ccdd86aa2625153b0a9528e41bdaa8c5a0edf9f0c003: Status 404 returned error can't find the container with id 6720e815f949672c4f99ccdd86aa2625153b0a9528e41bdaa8c5a0edf9f0c003
May 25 13:20:15.420827 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:15.420787    2559 generic.go:358] "Generic (PLEG): container finished" podID="3c387957-a263-450b-b5df-e541c32f1c9d" containerID="ab4ff4d93e5f11d7e4c78fef69ac609e23b4fa6cf889385cfbb1fb7fbc7ea0fb" exitCode=0
May 25 13:20:15.421078 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:15.420869    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2" event={"ID":"3c387957-a263-450b-b5df-e541c32f1c9d","Type":"ContainerDied","Data":"ab4ff4d93e5f11d7e4c78fef69ac609e23b4fa6cf889385cfbb1fb7fbc7ea0fb"}
May 25 13:20:15.421078 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:15.420904    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2" event={"ID":"3c387957-a263-450b-b5df-e541c32f1c9d","Type":"ContainerStarted","Data":"6720e815f949672c4f99ccdd86aa2625153b0a9528e41bdaa8c5a0edf9f0c003"}
May 25 13:20:18.432616 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:18.432573    2559 generic.go:358] "Generic (PLEG): container finished" podID="3c387957-a263-450b-b5df-e541c32f1c9d" containerID="03749371d3122cfa87d604e95519f6e354b6a0719b77636418fccd2a500b393e" exitCode=0
May 25 13:20:18.433050 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:18.432666    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2" event={"ID":"3c387957-a263-450b-b5df-e541c32f1c9d","Type":"ContainerDied","Data":"03749371d3122cfa87d604e95519f6e354b6a0719b77636418fccd2a500b393e"}
May 25 13:20:19.437735 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:19.437696    2559 generic.go:358] "Generic (PLEG): container finished" podID="3c387957-a263-450b-b5df-e541c32f1c9d" containerID="6baa1f81ecd5906e741df28b8c2edd21d6fe9311a098cfaeaf772793e2ee1e10" exitCode=0
May 25 13:20:19.438115 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:19.437752    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2" event={"ID":"3c387957-a263-450b-b5df-e541c32f1c9d","Type":"ContainerDied","Data":"6baa1f81ecd5906e741df28b8c2edd21d6fe9311a098cfaeaf772793e2ee1e10"}
May 25 13:20:20.568341 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:20.568316    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"
May 25 13:20:20.657209 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:20.657167    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c387957-a263-450b-b5df-e541c32f1c9d-bundle\") pod \"3c387957-a263-450b-b5df-e541c32f1c9d\" (UID: \"3c387957-a263-450b-b5df-e541c32f1c9d\") "
May 25 13:20:20.657401 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:20.657295    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c387957-a263-450b-b5df-e541c32f1c9d-util\") pod \"3c387957-a263-450b-b5df-e541c32f1c9d\" (UID: \"3c387957-a263-450b-b5df-e541c32f1c9d\") "
May 25 13:20:20.657401 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:20.657337    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbdpd\" (UniqueName: \"kubernetes.io/projected/3c387957-a263-450b-b5df-e541c32f1c9d-kube-api-access-zbdpd\") pod \"3c387957-a263-450b-b5df-e541c32f1c9d\" (UID: \"3c387957-a263-450b-b5df-e541c32f1c9d\") "
May 25 13:20:20.657573 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:20.657551    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c387957-a263-450b-b5df-e541c32f1c9d-bundle" (OuterVolumeSpecName: "bundle") pod "3c387957-a263-450b-b5df-e541c32f1c9d" (UID: "3c387957-a263-450b-b5df-e541c32f1c9d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:20:20.659386 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:20.659351    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c387957-a263-450b-b5df-e541c32f1c9d-kube-api-access-zbdpd" (OuterVolumeSpecName: "kube-api-access-zbdpd") pod "3c387957-a263-450b-b5df-e541c32f1c9d" (UID: "3c387957-a263-450b-b5df-e541c32f1c9d"). InnerVolumeSpecName "kube-api-access-zbdpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:20:20.661603 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:20.661575    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c387957-a263-450b-b5df-e541c32f1c9d-util" (OuterVolumeSpecName: "util") pod "3c387957-a263-450b-b5df-e541c32f1c9d" (UID: "3c387957-a263-450b-b5df-e541c32f1c9d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:20:20.758459 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:20.758425    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c387957-a263-450b-b5df-e541c32f1c9d-util\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:20:20.758597 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:20.758462    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbdpd\" (UniqueName: \"kubernetes.io/projected/3c387957-a263-450b-b5df-e541c32f1c9d-kube-api-access-zbdpd\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:20:20.758597 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:20.758477    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c387957-a263-450b-b5df-e541c32f1c9d-bundle\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:20:21.445655 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:21.445622    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2" event={"ID":"3c387957-a263-450b-b5df-e541c32f1c9d","Type":"ContainerDied","Data":"6720e815f949672c4f99ccdd86aa2625153b0a9528e41bdaa8c5a0edf9f0c003"}
May 25 13:20:21.445655 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:21.445650    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv25h2"
May 25 13:20:21.445892 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:21.445653    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6720e815f949672c4f99ccdd86aa2625153b0a9528e41bdaa8c5a0edf9f0c003"
May 25 13:20:37.402739 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.402680    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"]
May 25 13:20:37.403307 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.403282    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c387957-a263-450b-b5df-e541c32f1c9d" containerName="extract"
May 25 13:20:37.403510 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.403481    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c387957-a263-450b-b5df-e541c32f1c9d" containerName="extract"
May 25 13:20:37.403602 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.403542    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c387957-a263-450b-b5df-e541c32f1c9d" containerName="pull"
May 25 13:20:37.403602 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.403555    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c387957-a263-450b-b5df-e541c32f1c9d" containerName="pull"
May 25 13:20:37.403602 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.403576    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c387957-a263-450b-b5df-e541c32f1c9d" containerName="util"
May 25 13:20:37.403602 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.403591    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c387957-a263-450b-b5df-e541c32f1c9d" containerName="util"
May 25 13:20:37.403818 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.403759    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="3c387957-a263-450b-b5df-e541c32f1c9d" containerName="extract"
May 25 13:20:37.411684 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.411661    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"
May 25 13:20:37.413069 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.413049    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"]
May 25 13:20:37.414229 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.414208    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 25 13:20:37.414881 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.414860    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bxf9p\""
May 25 13:20:37.415004 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.414901    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 25 13:20:37.481513 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.481473    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/641a81d2-31fd-4806-b08f-37b817e6256e-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq\" (UID: \"641a81d2-31fd-4806-b08f-37b817e6256e\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"
May 25 13:20:37.481513 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.481517    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7hbv\" (UniqueName: \"kubernetes.io/projected/641a81d2-31fd-4806-b08f-37b817e6256e-kube-api-access-t7hbv\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq\" (UID: \"641a81d2-31fd-4806-b08f-37b817e6256e\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"
May 25 13:20:37.481784 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.481628    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/641a81d2-31fd-4806-b08f-37b817e6256e-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq\" (UID: \"641a81d2-31fd-4806-b08f-37b817e6256e\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"
May 25 13:20:37.583002 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.582957    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/641a81d2-31fd-4806-b08f-37b817e6256e-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq\" (UID: \"641a81d2-31fd-4806-b08f-37b817e6256e\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"
May 25 13:20:37.583002 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.583004    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t7hbv\" (UniqueName: \"kubernetes.io/projected/641a81d2-31fd-4806-b08f-37b817e6256e-kube-api-access-t7hbv\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq\" (UID: \"641a81d2-31fd-4806-b08f-37b817e6256e\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"
May 25 13:20:37.583256 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.583065    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/641a81d2-31fd-4806-b08f-37b817e6256e-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq\" (UID: \"641a81d2-31fd-4806-b08f-37b817e6256e\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"
May 25 13:20:37.583346 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.583323    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/641a81d2-31fd-4806-b08f-37b817e6256e-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq\" (UID: \"641a81d2-31fd-4806-b08f-37b817e6256e\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"
May 25 13:20:37.583456 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.583438    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/641a81d2-31fd-4806-b08f-37b817e6256e-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq\" (UID: \"641a81d2-31fd-4806-b08f-37b817e6256e\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"
May 25 13:20:37.590953 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.590932    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7hbv\" (UniqueName: \"kubernetes.io/projected/641a81d2-31fd-4806-b08f-37b817e6256e-kube-api-access-t7hbv\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq\" (UID: \"641a81d2-31fd-4806-b08f-37b817e6256e\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"
May 25 13:20:37.721637 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.721540    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"
May 25 13:20:37.845575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:37.845439    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"]
May 25 13:20:37.848205 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:20:37.848171    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod641a81d2_31fd_4806_b08f_37b817e6256e.slice/crio-40b60d8d78d0ee3d36fc5fd01fb4891173574f4b2ec045ad4bf609ccaf2f5386 WatchSource:0}: Error finding container 40b60d8d78d0ee3d36fc5fd01fb4891173574f4b2ec045ad4bf609ccaf2f5386: Status 404 returned error can't find the container with id 40b60d8d78d0ee3d36fc5fd01fb4891173574f4b2ec045ad4bf609ccaf2f5386
May 25 13:20:38.498572 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:38.498531    2559 generic.go:358] "Generic (PLEG): container finished" podID="641a81d2-31fd-4806-b08f-37b817e6256e" containerID="fde6fca807e502b48d1d791f188a8472cbc77b70ea2fe9a80d272cc2602d3c04" exitCode=0
May 25 13:20:38.499034 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:38.498617    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq" event={"ID":"641a81d2-31fd-4806-b08f-37b817e6256e","Type":"ContainerDied","Data":"fde6fca807e502b48d1d791f188a8472cbc77b70ea2fe9a80d272cc2602d3c04"}
May 25 13:20:38.499034 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:38.498657    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq" event={"ID":"641a81d2-31fd-4806-b08f-37b817e6256e","Type":"ContainerStarted","Data":"40b60d8d78d0ee3d36fc5fd01fb4891173574f4b2ec045ad4bf609ccaf2f5386"}
May 25 13:20:40.507178 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:40.507147    2559 generic.go:358] "Generic (PLEG): container finished" podID="641a81d2-31fd-4806-b08f-37b817e6256e" containerID="9030bf9459c4c12b8fe49000ce8f7abef6f71d694c430aa9d15b5ffc0d30cae5" exitCode=0
May 25 13:20:40.507572 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:40.507228    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq" event={"ID":"641a81d2-31fd-4806-b08f-37b817e6256e","Type":"ContainerDied","Data":"9030bf9459c4c12b8fe49000ce8f7abef6f71d694c430aa9d15b5ffc0d30cae5"}
May 25 13:20:41.512457 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:41.512419    2559 generic.go:358] "Generic (PLEG): container finished" podID="641a81d2-31fd-4806-b08f-37b817e6256e" containerID="70067b01797a44d19da3895fd345e545a998fdf2a16bd1747d6d728a4dcd458d" exitCode=0
May 25 13:20:41.512822 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:41.512493    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq" event={"ID":"641a81d2-31fd-4806-b08f-37b817e6256e","Type":"ContainerDied","Data":"70067b01797a44d19da3895fd345e545a998fdf2a16bd1747d6d728a4dcd458d"}
May 25 13:20:42.642951 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:42.642929    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"
May 25 13:20:42.725439 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:42.725405    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7hbv\" (UniqueName: \"kubernetes.io/projected/641a81d2-31fd-4806-b08f-37b817e6256e-kube-api-access-t7hbv\") pod \"641a81d2-31fd-4806-b08f-37b817e6256e\" (UID: \"641a81d2-31fd-4806-b08f-37b817e6256e\") "
May 25 13:20:42.725633 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:42.725452    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/641a81d2-31fd-4806-b08f-37b817e6256e-bundle\") pod \"641a81d2-31fd-4806-b08f-37b817e6256e\" (UID: \"641a81d2-31fd-4806-b08f-37b817e6256e\") "
May 25 13:20:42.725633 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:42.725498    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/641a81d2-31fd-4806-b08f-37b817e6256e-util\") pod \"641a81d2-31fd-4806-b08f-37b817e6256e\" (UID: \"641a81d2-31fd-4806-b08f-37b817e6256e\") "
May 25 13:20:42.726637 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:42.726608    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/641a81d2-31fd-4806-b08f-37b817e6256e-bundle" (OuterVolumeSpecName: "bundle") pod "641a81d2-31fd-4806-b08f-37b817e6256e" (UID: "641a81d2-31fd-4806-b08f-37b817e6256e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:20:42.727545 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:42.727523    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/641a81d2-31fd-4806-b08f-37b817e6256e-kube-api-access-t7hbv" (OuterVolumeSpecName: "kube-api-access-t7hbv") pod "641a81d2-31fd-4806-b08f-37b817e6256e" (UID: "641a81d2-31fd-4806-b08f-37b817e6256e"). InnerVolumeSpecName "kube-api-access-t7hbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:20:42.731611 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:42.731592    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/641a81d2-31fd-4806-b08f-37b817e6256e-util" (OuterVolumeSpecName: "util") pod "641a81d2-31fd-4806-b08f-37b817e6256e" (UID: "641a81d2-31fd-4806-b08f-37b817e6256e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:20:42.826151 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:42.826056    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7hbv\" (UniqueName: \"kubernetes.io/projected/641a81d2-31fd-4806-b08f-37b817e6256e-kube-api-access-t7hbv\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:20:42.826151 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:42.826099    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/641a81d2-31fd-4806-b08f-37b817e6256e-bundle\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:20:42.826151 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:42.826108    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/641a81d2-31fd-4806-b08f-37b817e6256e-util\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:20:43.520667 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:43.520629    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq" event={"ID":"641a81d2-31fd-4806-b08f-37b817e6256e","Type":"ContainerDied","Data":"40b60d8d78d0ee3d36fc5fd01fb4891173574f4b2ec045ad4bf609ccaf2f5386"}
May 25 13:20:43.520667 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:43.520665    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40b60d8d78d0ee3d36fc5fd01fb4891173574f4b2ec045ad4bf609ccaf2f5386"
May 25 13:20:43.520667 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:43.520638    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835pn8hq"
May 25 13:20:52.436691 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.436657    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"]
May 25 13:20:52.437084 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.436951    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="641a81d2-31fd-4806-b08f-37b817e6256e" containerName="util"
May 25 13:20:52.437084 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.436963    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="641a81d2-31fd-4806-b08f-37b817e6256e" containerName="util"
May 25 13:20:52.437084 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.436978    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="641a81d2-31fd-4806-b08f-37b817e6256e" containerName="extract"
May 25 13:20:52.437084 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.436984    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="641a81d2-31fd-4806-b08f-37b817e6256e" containerName="extract"
May 25 13:20:52.437084 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.436991    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="641a81d2-31fd-4806-b08f-37b817e6256e" containerName="pull"
May 25 13:20:52.437084 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.436996    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="641a81d2-31fd-4806-b08f-37b817e6256e" containerName="pull"
May 25 13:20:52.437084 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.437042    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="641a81d2-31fd-4806-b08f-37b817e6256e" containerName="extract"
May 25 13:20:52.439935 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.439919    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"
May 25 13:20:52.442331 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.442311    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bxf9p\""
May 25 13:20:52.442657 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.442640    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 25 13:20:52.443064 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.443049    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 25 13:20:52.452427 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.452408    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"]
May 25 13:20:52.501733 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.501685    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4fb4e7c-f9f6-4054-8f77-04f681038888-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc\" (UID: \"e4fb4e7c-f9f6-4054-8f77-04f681038888\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"
May 25 13:20:52.501867 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.501762    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s28ns\" (UniqueName: \"kubernetes.io/projected/e4fb4e7c-f9f6-4054-8f77-04f681038888-kube-api-access-s28ns\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc\" (UID: \"e4fb4e7c-f9f6-4054-8f77-04f681038888\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"
May 25 13:20:52.501867 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.501785    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4fb4e7c-f9f6-4054-8f77-04f681038888-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc\" (UID: \"e4fb4e7c-f9f6-4054-8f77-04f681038888\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"
May 25 13:20:52.602838 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.602804    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4fb4e7c-f9f6-4054-8f77-04f681038888-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc\" (UID: \"e4fb4e7c-f9f6-4054-8f77-04f681038888\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"
May 25 13:20:52.602997 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.602863    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s28ns\" (UniqueName: \"kubernetes.io/projected/e4fb4e7c-f9f6-4054-8f77-04f681038888-kube-api-access-s28ns\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc\" (UID: \"e4fb4e7c-f9f6-4054-8f77-04f681038888\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"
May 25 13:20:52.602997 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.602892    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4fb4e7c-f9f6-4054-8f77-04f681038888-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc\" (UID: \"e4fb4e7c-f9f6-4054-8f77-04f681038888\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"
May 25 13:20:52.603184 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.603165    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4fb4e7c-f9f6-4054-8f77-04f681038888-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc\" (UID: \"e4fb4e7c-f9f6-4054-8f77-04f681038888\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"
May 25 13:20:52.603245 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.603225    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4fb4e7c-f9f6-4054-8f77-04f681038888-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc\" (UID: \"e4fb4e7c-f9f6-4054-8f77-04f681038888\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"
May 25 13:20:52.611628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.611608    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s28ns\" (UniqueName: \"kubernetes.io/projected/e4fb4e7c-f9f6-4054-8f77-04f681038888-kube-api-access-s28ns\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc\" (UID: \"e4fb4e7c-f9f6-4054-8f77-04f681038888\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"
May 25 13:20:52.749209 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.749181    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"
May 25 13:20:52.886320 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:52.886294    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"]
May 25 13:20:52.887827 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:20:52.887793    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4fb4e7c_f9f6_4054_8f77_04f681038888.slice/crio-c2b10144d6d22499c7125733c6785ac5b98d2c500038c7cff89ad95341decc36 WatchSource:0}: Error finding container c2b10144d6d22499c7125733c6785ac5b98d2c500038c7cff89ad95341decc36: Status 404 returned error can't find the container with id c2b10144d6d22499c7125733c6785ac5b98d2c500038c7cff89ad95341decc36
May 25 13:20:53.552885 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:53.552855    2559 generic.go:358] "Generic (PLEG): container finished" podID="e4fb4e7c-f9f6-4054-8f77-04f681038888" containerID="9699aba655fc9493b0556f8c71644703b593cb974c686c6094c7e9ffc018f7ce" exitCode=0
May 25 13:20:53.553303 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:53.552941    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc" event={"ID":"e4fb4e7c-f9f6-4054-8f77-04f681038888","Type":"ContainerDied","Data":"9699aba655fc9493b0556f8c71644703b593cb974c686c6094c7e9ffc018f7ce"}
May 25 13:20:53.553303 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:53.552984    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc" event={"ID":"e4fb4e7c-f9f6-4054-8f77-04f681038888","Type":"ContainerStarted","Data":"c2b10144d6d22499c7125733c6785ac5b98d2c500038c7cff89ad95341decc36"}
May 25 13:20:54.166943 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.166905    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-mt99k"]
May 25 13:20:54.170006 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.169988    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-mt99k"
May 25 13:20:54.175430 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.175278    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\""
May 25 13:20:54.175560 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.175488    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-xfz8f\""
May 25 13:20:54.175560 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.175526    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\""
May 25 13:20:54.193461 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.193434    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-mt99k"]
May 25 13:20:54.216918 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.216892    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjltt\" (UniqueName: \"kubernetes.io/projected/c592bdba-cc51-4d9c-8b1f-32b218e4d291-kube-api-access-xjltt\") pod \"servicemesh-operator3-55f49c5f94-mt99k\" (UID: \"c592bdba-cc51-4d9c-8b1f-32b218e4d291\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-mt99k"
May 25 13:20:54.217040 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.216934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/c592bdba-cc51-4d9c-8b1f-32b218e4d291-operator-config\") pod \"servicemesh-operator3-55f49c5f94-mt99k\" (UID: \"c592bdba-cc51-4d9c-8b1f-32b218e4d291\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-mt99k"
May 25 13:20:54.317919 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.317880    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xjltt\" (UniqueName: \"kubernetes.io/projected/c592bdba-cc51-4d9c-8b1f-32b218e4d291-kube-api-access-xjltt\") pod \"servicemesh-operator3-55f49c5f94-mt99k\" (UID: \"c592bdba-cc51-4d9c-8b1f-32b218e4d291\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-mt99k"
May 25 13:20:54.318063 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.317926    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/c592bdba-cc51-4d9c-8b1f-32b218e4d291-operator-config\") pod \"servicemesh-operator3-55f49c5f94-mt99k\" (UID: \"c592bdba-cc51-4d9c-8b1f-32b218e4d291\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-mt99k"
May 25 13:20:54.320255 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.320225    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/c592bdba-cc51-4d9c-8b1f-32b218e4d291-operator-config\") pod \"servicemesh-operator3-55f49c5f94-mt99k\" (UID: \"c592bdba-cc51-4d9c-8b1f-32b218e4d291\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-mt99k"
May 25 13:20:54.338009 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.337980    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjltt\" (UniqueName: \"kubernetes.io/projected/c592bdba-cc51-4d9c-8b1f-32b218e4d291-kube-api-access-xjltt\") pod \"servicemesh-operator3-55f49c5f94-mt99k\" (UID: \"c592bdba-cc51-4d9c-8b1f-32b218e4d291\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-mt99k"
May 25 13:20:54.483900 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.483815    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-mt99k"
May 25 13:20:54.619216 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:54.619182    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-mt99k"]
May 25 13:20:54.622553 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:20:54.622518    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc592bdba_cc51_4d9c_8b1f_32b218e4d291.slice/crio-45e7342e81cce8223573fa0e4211199b9292e65af166a8920d70e27ef1ef44d7 WatchSource:0}: Error finding container 45e7342e81cce8223573fa0e4211199b9292e65af166a8920d70e27ef1ef44d7: Status 404 returned error can't find the container with id 45e7342e81cce8223573fa0e4211199b9292e65af166a8920d70e27ef1ef44d7
May 25 13:20:55.562798 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:55.562693    2559 generic.go:358] "Generic (PLEG): container finished" podID="e4fb4e7c-f9f6-4054-8f77-04f681038888" containerID="03b5f9ce13bec9ef6b6a307c7d802cef6d721600b3cf92c1d8d8201d215dc11e" exitCode=0
May 25 13:20:55.562798 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:55.562753    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc" event={"ID":"e4fb4e7c-f9f6-4054-8f77-04f681038888","Type":"ContainerDied","Data":"03b5f9ce13bec9ef6b6a307c7d802cef6d721600b3cf92c1d8d8201d215dc11e"}
May 25 13:20:55.564544 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:55.564508    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-mt99k" event={"ID":"c592bdba-cc51-4d9c-8b1f-32b218e4d291","Type":"ContainerStarted","Data":"45e7342e81cce8223573fa0e4211199b9292e65af166a8920d70e27ef1ef44d7"}
May 25 13:20:56.569934 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:56.569894    2559 generic.go:358] "Generic (PLEG): container finished" podID="e4fb4e7c-f9f6-4054-8f77-04f681038888" containerID="b7dbe5f290f4a01636c42fc35f241b042dc9ccea695492debb0ce050ab9b3b12" exitCode=0
May 25 13:20:56.570329 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:56.569954    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc" event={"ID":"e4fb4e7c-f9f6-4054-8f77-04f681038888","Type":"ContainerDied","Data":"b7dbe5f290f4a01636c42fc35f241b042dc9ccea695492debb0ce050ab9b3b12"}
May 25 13:20:57.575217 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:57.575180    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-mt99k" event={"ID":"c592bdba-cc51-4d9c-8b1f-32b218e4d291","Type":"ContainerStarted","Data":"49dab1d0c2265497c6ec7c431df478b347874f24010c214af8178596d31778da"}
May 25 13:20:57.575611 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:57.575243    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-mt99k"
May 25 13:20:57.602830 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:57.602780    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-mt99k" podStartSLOduration=1.128735525 podStartE2EDuration="3.602764839s" podCreationTimestamp="2026-05-25 13:20:54 +0000 UTC" firstStartedPulling="2026-05-25 13:20:54.625105153 +0000 UTC m=+921.344068273" lastFinishedPulling="2026-05-25 13:20:57.099134453 +0000 UTC m=+923.818097587" observedRunningTime="2026-05-25 13:20:57.600265548 +0000 UTC m=+924.319228688" watchObservedRunningTime="2026-05-25 13:20:57.602764839 +0000 UTC m=+924.321727977"
May 25 13:20:57.710544 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:57.710517    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"
May 25 13:20:57.850558 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:57.850473    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4fb4e7c-f9f6-4054-8f77-04f681038888-bundle\") pod \"e4fb4e7c-f9f6-4054-8f77-04f681038888\" (UID: \"e4fb4e7c-f9f6-4054-8f77-04f681038888\") "
May 25 13:20:57.850558 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:57.850545    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s28ns\" (UniqueName: \"kubernetes.io/projected/e4fb4e7c-f9f6-4054-8f77-04f681038888-kube-api-access-s28ns\") pod \"e4fb4e7c-f9f6-4054-8f77-04f681038888\" (UID: \"e4fb4e7c-f9f6-4054-8f77-04f681038888\") "
May 25 13:20:57.850820 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:57.850570    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4fb4e7c-f9f6-4054-8f77-04f681038888-util\") pod \"e4fb4e7c-f9f6-4054-8f77-04f681038888\" (UID: \"e4fb4e7c-f9f6-4054-8f77-04f681038888\") "
May 25 13:20:57.851822 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:57.851785    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4fb4e7c-f9f6-4054-8f77-04f681038888-bundle" (OuterVolumeSpecName: "bundle") pod "e4fb4e7c-f9f6-4054-8f77-04f681038888" (UID: "e4fb4e7c-f9f6-4054-8f77-04f681038888"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:20:57.852612 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:57.852588    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4fb4e7c-f9f6-4054-8f77-04f681038888-kube-api-access-s28ns" (OuterVolumeSpecName: "kube-api-access-s28ns") pod "e4fb4e7c-f9f6-4054-8f77-04f681038888" (UID: "e4fb4e7c-f9f6-4054-8f77-04f681038888"). InnerVolumeSpecName "kube-api-access-s28ns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:20:57.859372 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:57.859344    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4fb4e7c-f9f6-4054-8f77-04f681038888-util" (OuterVolumeSpecName: "util") pod "e4fb4e7c-f9f6-4054-8f77-04f681038888" (UID: "e4fb4e7c-f9f6-4054-8f77-04f681038888"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:20:57.951488 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:57.951459    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4fb4e7c-f9f6-4054-8f77-04f681038888-bundle\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:20:57.951639 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:57.951503    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s28ns\" (UniqueName: \"kubernetes.io/projected/e4fb4e7c-f9f6-4054-8f77-04f681038888-kube-api-access-s28ns\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:20:57.951639 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:57.951526    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4fb4e7c-f9f6-4054-8f77-04f681038888-util\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:20:58.580439 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:58.580401    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc" event={"ID":"e4fb4e7c-f9f6-4054-8f77-04f681038888","Type":"ContainerDied","Data":"c2b10144d6d22499c7125733c6785ac5b98d2c500038c7cff89ad95341decc36"}
May 25 13:20:58.580439 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:58.580440    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2b10144d6d22499c7125733c6785ac5b98d2c500038c7cff89ad95341decc36"
May 25 13:20:58.580439 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:20:58.580421    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762adv9gc"
May 25 13:21:03.005926 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.005891    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"]
May 25 13:21:03.006353 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.006328    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4fb4e7c-f9f6-4054-8f77-04f681038888" containerName="extract"
May 25 13:21:03.006353 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.006345    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4fb4e7c-f9f6-4054-8f77-04f681038888" containerName="extract"
May 25 13:21:03.006474 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.006361    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4fb4e7c-f9f6-4054-8f77-04f681038888" containerName="util"
May 25 13:21:03.006474 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.006369    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4fb4e7c-f9f6-4054-8f77-04f681038888" containerName="util"
May 25 13:21:03.006474 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.006380    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4fb4e7c-f9f6-4054-8f77-04f681038888" containerName="pull"
May 25 13:21:03.006474 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.006388    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4fb4e7c-f9f6-4054-8f77-04f681038888" containerName="pull"
May 25 13:21:03.006474 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.006472    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="e4fb4e7c-f9f6-4054-8f77-04f681038888" containerName="extract"
May 25 13:21:03.010568 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.010550    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.017110 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.016903    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
May 25 13:21:03.017110 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.017067    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 25 13:21:03.017410 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.017382    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
May 25 13:21:03.017567 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.017531    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6jqnl\""
May 25 13:21:03.017965 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.017950    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 25 13:21:03.018065 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.018049    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
May 25 13:21:03.018569 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.018557    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
May 25 13:21:03.028084 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.028061    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"]
May 25 13:21:03.097022 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.096987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cs5r\" (UniqueName: \"kubernetes.io/projected/b95365e2-13be-4cf6-ae39-b13c23078710-kube-api-access-2cs5r\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.097186 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.097038    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b95365e2-13be-4cf6-ae39-b13c23078710-local-certs\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.097186 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.097056    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-cacerts\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.097186 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.097086    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b95365e2-13be-4cf6-ae39-b13c23078710-istio-token\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.097186 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.097127    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-istio-kubeconfig\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.097186 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.097148    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.097186 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.097164    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b95365e2-13be-4cf6-ae39-b13c23078710-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.198030 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.198000    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2cs5r\" (UniqueName: \"kubernetes.io/projected/b95365e2-13be-4cf6-ae39-b13c23078710-kube-api-access-2cs5r\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.198190 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.198054    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b95365e2-13be-4cf6-ae39-b13c23078710-local-certs\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.198190 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.198146    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-cacerts\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.198190 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.198166    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b95365e2-13be-4cf6-ae39-b13c23078710-istio-token\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.198303 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.198206    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-istio-kubeconfig\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.198303 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.198230    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.198397 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.198340    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b95365e2-13be-4cf6-ae39-b13c23078710-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.198978 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.198951    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b95365e2-13be-4cf6-ae39-b13c23078710-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.200675 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.200643    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-cacerts\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.200788 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.200744    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-istio-kubeconfig\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.200891 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.200873    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b95365e2-13be-4cf6-ae39-b13c23078710-local-certs\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.200924 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.200907    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.207596 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.207569    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b95365e2-13be-4cf6-ae39-b13c23078710-istio-token\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.207867 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.207841    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cs5r\" (UniqueName: \"kubernetes.io/projected/b95365e2-13be-4cf6-ae39-b13c23078710-kube-api-access-2cs5r\") pod \"istiod-openshift-gateway-767cf5cf55-hn88z\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") " pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.320538 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.319949    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:03.486304 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.486179    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"]
May 25 13:21:03.488640 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:21:03.488610    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb95365e2_13be_4cf6_ae39_b13c23078710.slice/crio-0c55f9932e5fbefef933a059cd2425e3270157fbfb4554e4a63eb85d2d7181a5 WatchSource:0}: Error finding container 0c55f9932e5fbefef933a059cd2425e3270157fbfb4554e4a63eb85d2d7181a5: Status 404 returned error can't find the container with id 0c55f9932e5fbefef933a059cd2425e3270157fbfb4554e4a63eb85d2d7181a5
May 25 13:21:03.599997 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:03.599904    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z" event={"ID":"b95365e2-13be-4cf6-ae39-b13c23078710","Type":"ContainerStarted","Data":"0c55f9932e5fbefef933a059cd2425e3270157fbfb4554e4a63eb85d2d7181a5"}
May 25 13:21:06.096275 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:06.096239    2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 13:21:06.096607 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:06.096318    2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 13:21:06.614788 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:06.614748    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z" event={"ID":"b95365e2-13be-4cf6-ae39-b13c23078710","Type":"ContainerStarted","Data":"44931a755fc4dc0abe07d26f5cf06044491f84094c43d24761d6ac2a234783e7"}
May 25 13:21:06.615795 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:06.615773    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:06.642518 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:06.642447    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z" podStartSLOduration=2.03745413 podStartE2EDuration="4.642427247s" podCreationTimestamp="2026-05-25 13:21:02 +0000 UTC" firstStartedPulling="2026-05-25 13:21:03.491066559 +0000 UTC m=+930.210029679" lastFinishedPulling="2026-05-25 13:21:06.096039676 +0000 UTC m=+932.815002796" observedRunningTime="2026-05-25 13:21:06.637365145 +0000 UTC m=+933.356328285" watchObservedRunningTime="2026-05-25 13:21:06.642427247 +0000 UTC m=+933.361390388"
May 25 13:21:07.619729 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:07.619685    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:21:08.582383 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:08.582353    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-mt99k"
May 25 13:21:08.967261 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:08.967101    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"]
May 25 13:21:08.970982 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:08.970949    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:08.975387 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:08.975354    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-55ttb\""
May 25 13:21:08.990803 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:08.990770    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"]
May 25 13:21:09.051514 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.051482    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istio-token\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.051735 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.051529    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.051735 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.051570    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-credential-socket\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.051735 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.051619    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istio-data\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.051735 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.051646    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istio-envoy\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.051735 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.051689    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.051939 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.051797    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-workload-socket\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.051939 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.051849    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7smvx\" (UniqueName: \"kubernetes.io/projected/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-kube-api-access-7smvx\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.051939 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.051889    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-workload-certs\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.153061 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.153025    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-credential-socket\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.153061 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.153063    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istio-data\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.153301 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.153084    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istio-envoy\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.153301 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.153112    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.153301 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.153150    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-workload-socket\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.153301 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.153173    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7smvx\" (UniqueName: \"kubernetes.io/projected/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-kube-api-access-7smvx\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.153301 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.153202    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-workload-certs\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.153301 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.153243    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istio-token\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.153301 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.153275    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.153653 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.153478    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istio-data\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.153653 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.153546    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-credential-socket\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.153653 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.153591    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-workload-socket\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.153846 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.153776    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-workload-certs\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.154126 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.154105    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.155446 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.155417    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istio-envoy\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.155637 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.155621    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.165601 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.165576    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-istio-token\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.166099 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.166071    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7smvx\" (UniqueName: \"kubernetes.io/projected/4f70df2d-eb18-4ee6-b889-03a6fbbd5081-kube-api-access-7smvx\") pod \"openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw\" (UID: \"4f70df2d-eb18-4ee6-b889-03a6fbbd5081\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.284184 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.284096    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:09.434263 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.434218    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"]
May 25 13:21:09.438344 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:21:09.438316    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f70df2d_eb18_4ee6_b889_03a6fbbd5081.slice/crio-9e15ad6d4a78e3c82596bc7177453b03836d5ecb5a2250e72640961321b3af74 WatchSource:0}: Error finding container 9e15ad6d4a78e3c82596bc7177453b03836d5ecb5a2250e72640961321b3af74: Status 404 returned error can't find the container with id 9e15ad6d4a78e3c82596bc7177453b03836d5ecb5a2250e72640961321b3af74
May 25 13:21:09.625500 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:09.625413    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw" event={"ID":"4f70df2d-eb18-4ee6-b889-03a6fbbd5081","Type":"ContainerStarted","Data":"9e15ad6d4a78e3c82596bc7177453b03836d5ecb5a2250e72640961321b3af74"}
May 25 13:21:12.153049 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:12.153011    2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 13:21:12.153311 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:12.153081    2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 13:21:12.153311 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:12.153112    2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 13:21:12.647489 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:12.647450    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw" event={"ID":"4f70df2d-eb18-4ee6-b889-03a6fbbd5081","Type":"ContainerStarted","Data":"53ca59b44f30c1d6c113d21c7cc3fe19be7f4809a248d25d84afed8953a7b136"}
May 25 13:21:12.679769 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:12.679708    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw" podStartSLOduration=1.9677313509999999 podStartE2EDuration="4.679693218s" podCreationTimestamp="2026-05-25 13:21:08 +0000 UTC" firstStartedPulling="2026-05-25 13:21:09.440784139 +0000 UTC m=+936.159747256" lastFinishedPulling="2026-05-25 13:21:12.152746002 +0000 UTC m=+938.871709123" observedRunningTime="2026-05-25 13:21:12.677092427 +0000 UTC m=+939.396055832" watchObservedRunningTime="2026-05-25 13:21:12.679693218 +0000 UTC m=+939.398656408"
May 25 13:21:13.284904 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:13.284863    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:13.289299 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:13.289276    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:13.650760 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:13.650728    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:13.651537 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:13.651517    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw"
May 25 13:21:19.580547 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.580508    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"]
May 25 13:21:19.585107 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.585085    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"
May 25 13:21:19.587564 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.587548    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 25 13:21:19.587690 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.587671    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-bxf9p\""
May 25 13:21:19.588712 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.588696    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 25 13:21:19.592275 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.592257    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"]
May 25 13:21:19.674135 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.674104    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"]
May 25 13:21:19.677477 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.677457    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"
May 25 13:21:19.685886 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.685861    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"]
May 25 13:21:19.741460 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.741430    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-bundle\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj\" (UID: \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"
May 25 13:21:19.741627 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.741483    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-util\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj\" (UID: \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"
May 25 13:21:19.741627 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.741535    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lmjl\" (UniqueName: \"kubernetes.io/projected/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-kube-api-access-9lmjl\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj\" (UID: \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"
May 25 13:21:19.780098 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.780065    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"]
May 25 13:21:19.783550 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.783533    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"
May 25 13:21:19.790818 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.790797    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"]
May 25 13:21:19.843012 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.842934    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwf8m\" (UniqueName: \"kubernetes.io/projected/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-kube-api-access-mwf8m\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx\" (UID: \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"
May 25 13:21:19.843012 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.842974    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-bundle\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj\" (UID: \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"
May 25 13:21:19.843185 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.843067    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-util\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj\" (UID: \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"
May 25 13:21:19.843185 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.843107    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx\" (UID: \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"
May 25 13:21:19.843185 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.843153    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9lmjl\" (UniqueName: \"kubernetes.io/projected/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-kube-api-access-9lmjl\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj\" (UID: \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"
May 25 13:21:19.843300 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.843188    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx\" (UID: \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"
May 25 13:21:19.843373 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.843359    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-bundle\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj\" (UID: \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"
May 25 13:21:19.843409 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.843387    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-util\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj\" (UID: \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"
May 25 13:21:19.851548 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.851519    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lmjl\" (UniqueName: \"kubernetes.io/projected/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-kube-api-access-9lmjl\") pod \"b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj\" (UID: \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\") " pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"
May 25 13:21:19.878243 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.878217    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"]
May 25 13:21:19.881581 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.881566    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"
May 25 13:21:19.890362 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.890338    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"]
May 25 13:21:19.894702 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.894684    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"
May 25 13:21:19.944078 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.944039    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mwf8m\" (UniqueName: \"kubernetes.io/projected/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-kube-api-access-mwf8m\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx\" (UID: \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"
May 25 13:21:19.944219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.944103    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b35ba0d-cc5a-4126-82f6-6d58f002f441-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6\" (UID: \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"
May 25 13:21:19.944219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.944145    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b35ba0d-cc5a-4126-82f6-6d58f002f441-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6\" (UID: \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"
May 25 13:21:19.944341 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.944284    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fcz4\" (UniqueName: \"kubernetes.io/projected/6b35ba0d-cc5a-4126-82f6-6d58f002f441-kube-api-access-4fcz4\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6\" (UID: \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"
May 25 13:21:19.944341 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.944327    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx\" (UID: \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"
May 25 13:21:19.944443 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.944411    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx\" (UID: \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"
May 25 13:21:19.944764 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.944740    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-util\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx\" (UID: \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"
May 25 13:21:19.944867 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.944827    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-bundle\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx\" (UID: \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"
May 25 13:21:19.952880 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.952848    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwf8m\" (UniqueName: \"kubernetes.io/projected/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-kube-api-access-mwf8m\") pod \"309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx\" (UID: \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\") " pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"
May 25 13:21:19.987179 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:19.987150    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"
May 25 13:21:20.025551 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.025381    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"]
May 25 13:21:20.027914 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:21:20.027883    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d209dfa_cd8e_4ecd_a441_002d7832fcc6.slice/crio-1b663b4fb67f66db475f522e37103521d16aa6920687851b9badd2fb7199f663 WatchSource:0}: Error finding container 1b663b4fb67f66db475f522e37103521d16aa6920687851b9badd2fb7199f663: Status 404 returned error can't find the container with id 1b663b4fb67f66db475f522e37103521d16aa6920687851b9badd2fb7199f663
May 25 13:21:20.045253 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.045220    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/626358cc-23d9-41ba-aa03-a04e27ba07e1-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4\" (UID: \"626358cc-23d9-41ba-aa03-a04e27ba07e1\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"
May 25 13:21:20.045807 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.045330    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4fcz4\" (UniqueName: \"kubernetes.io/projected/6b35ba0d-cc5a-4126-82f6-6d58f002f441-kube-api-access-4fcz4\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6\" (UID: \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"
May 25 13:21:20.045807 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.045771    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/626358cc-23d9-41ba-aa03-a04e27ba07e1-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4\" (UID: \"626358cc-23d9-41ba-aa03-a04e27ba07e1\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"
May 25 13:21:20.045973 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.045914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfxt7\" (UniqueName: \"kubernetes.io/projected/626358cc-23d9-41ba-aa03-a04e27ba07e1-kube-api-access-gfxt7\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4\" (UID: \"626358cc-23d9-41ba-aa03-a04e27ba07e1\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"
May 25 13:21:20.046112 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.046021    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b35ba0d-cc5a-4126-82f6-6d58f002f441-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6\" (UID: \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"
May 25 13:21:20.046112 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.046077    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b35ba0d-cc5a-4126-82f6-6d58f002f441-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6\" (UID: \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"
May 25 13:21:20.046710 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.046660    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b35ba0d-cc5a-4126-82f6-6d58f002f441-bundle\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6\" (UID: \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"
May 25 13:21:20.046910 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.046886    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b35ba0d-cc5a-4126-82f6-6d58f002f441-util\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6\" (UID: \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"
May 25 13:21:20.054175 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.054154    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fcz4\" (UniqueName: \"kubernetes.io/projected/6b35ba0d-cc5a-4126-82f6-6d58f002f441-kube-api-access-4fcz4\") pod \"ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6\" (UID: \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\") " pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"
May 25 13:21:20.092885 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.092855    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"
May 25 13:21:20.126193 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.126157    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"]
May 25 13:21:20.130251 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:21:20.130217    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17f85d51_37ce_46bf_a8f0_6b84ced1a30f.slice/crio-9c62ff10d7e6bfc67a20349e548b9771d29be5df5c2a3b87bc5f16c796a4af87 WatchSource:0}: Error finding container 9c62ff10d7e6bfc67a20349e548b9771d29be5df5c2a3b87bc5f16c796a4af87: Status 404 returned error can't find the container with id 9c62ff10d7e6bfc67a20349e548b9771d29be5df5c2a3b87bc5f16c796a4af87
May 25 13:21:20.147694 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.147138    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/626358cc-23d9-41ba-aa03-a04e27ba07e1-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4\" (UID: \"626358cc-23d9-41ba-aa03-a04e27ba07e1\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"
May 25 13:21:20.147694 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.147194    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/626358cc-23d9-41ba-aa03-a04e27ba07e1-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4\" (UID: \"626358cc-23d9-41ba-aa03-a04e27ba07e1\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"
May 25 13:21:20.147694 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.147265    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gfxt7\" (UniqueName: \"kubernetes.io/projected/626358cc-23d9-41ba-aa03-a04e27ba07e1-kube-api-access-gfxt7\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4\" (UID: \"626358cc-23d9-41ba-aa03-a04e27ba07e1\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"
May 25 13:21:20.147995 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.147781    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/626358cc-23d9-41ba-aa03-a04e27ba07e1-bundle\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4\" (UID: \"626358cc-23d9-41ba-aa03-a04e27ba07e1\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"
May 25 13:21:20.148325 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.148281    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/626358cc-23d9-41ba-aa03-a04e27ba07e1-util\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4\" (UID: \"626358cc-23d9-41ba-aa03-a04e27ba07e1\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"
May 25 13:21:20.156288 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.156223    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfxt7\" (UniqueName: \"kubernetes.io/projected/626358cc-23d9-41ba-aa03-a04e27ba07e1-kube-api-access-gfxt7\") pod \"d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4\" (UID: \"626358cc-23d9-41ba-aa03-a04e27ba07e1\") " pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"
May 25 13:21:20.191819 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.191785    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"
May 25 13:21:20.227489 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.227441    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"]
May 25 13:21:20.230029 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:21:20.230006    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b35ba0d_cc5a_4126_82f6_6d58f002f441.slice/crio-7e405432d29d2ba9e21eb55bb32223f53da863418c04ffc2a8b57bba7505ce3a WatchSource:0}: Error finding container 7e405432d29d2ba9e21eb55bb32223f53da863418c04ffc2a8b57bba7505ce3a: Status 404 returned error can't find the container with id 7e405432d29d2ba9e21eb55bb32223f53da863418c04ffc2a8b57bba7505ce3a
May 25 13:21:20.338982 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.338953    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"]
May 25 13:21:20.342981 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:21:20.342954    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod626358cc_23d9_41ba_aa03_a04e27ba07e1.slice/crio-2a422876a77bf261f493fe206303a096dec657783c12aed3f01259d316433c16 WatchSource:0}: Error finding container 2a422876a77bf261f493fe206303a096dec657783c12aed3f01259d316433c16: Status 404 returned error can't find the container with id 2a422876a77bf261f493fe206303a096dec657783c12aed3f01259d316433c16
May 25 13:21:20.672545 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.672458    2559 generic.go:358] "Generic (PLEG): container finished" podID="626358cc-23d9-41ba-aa03-a04e27ba07e1" containerID="a58ec5125c420d5a602a6841bd37ca5c54dc775d260c9668440093e0924c7973" exitCode=0
May 25 13:21:20.672545 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.672526    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4" event={"ID":"626358cc-23d9-41ba-aa03-a04e27ba07e1","Type":"ContainerDied","Data":"a58ec5125c420d5a602a6841bd37ca5c54dc775d260c9668440093e0924c7973"}
May 25 13:21:20.672994 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.672551    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4" event={"ID":"626358cc-23d9-41ba-aa03-a04e27ba07e1","Type":"ContainerStarted","Data":"2a422876a77bf261f493fe206303a096dec657783c12aed3f01259d316433c16"}
May 25 13:21:20.674066 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.674046    2559 generic.go:358] "Generic (PLEG): container finished" podID="6b35ba0d-cc5a-4126-82f6-6d58f002f441" containerID="9b54a259e8ef40ec4a67ed33247a145ffa800c65ef7f08e04071ad906b4914fd" exitCode=0
May 25 13:21:20.674147 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.674111    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6" event={"ID":"6b35ba0d-cc5a-4126-82f6-6d58f002f441","Type":"ContainerDied","Data":"9b54a259e8ef40ec4a67ed33247a145ffa800c65ef7f08e04071ad906b4914fd"}
May 25 13:21:20.674193 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.674133    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6" event={"ID":"6b35ba0d-cc5a-4126-82f6-6d58f002f441","Type":"ContainerStarted","Data":"7e405432d29d2ba9e21eb55bb32223f53da863418c04ffc2a8b57bba7505ce3a"}
May 25 13:21:20.675430 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.675400    2559 generic.go:358] "Generic (PLEG): container finished" podID="17f85d51-37ce-46bf-a8f0-6b84ced1a30f" containerID="878937c483da1ee98808eb16cdf5958338b095d2dd191796472a5c0314975b85" exitCode=0
May 25 13:21:20.675491 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.675456    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx" event={"ID":"17f85d51-37ce-46bf-a8f0-6b84ced1a30f","Type":"ContainerDied","Data":"878937c483da1ee98808eb16cdf5958338b095d2dd191796472a5c0314975b85"}
May 25 13:21:20.675539 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.675496    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx" event={"ID":"17f85d51-37ce-46bf-a8f0-6b84ced1a30f","Type":"ContainerStarted","Data":"9c62ff10d7e6bfc67a20349e548b9771d29be5df5c2a3b87bc5f16c796a4af87"}
May 25 13:21:20.676956 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.676937    2559 generic.go:358] "Generic (PLEG): container finished" podID="8d209dfa-cd8e-4ecd-a441-002d7832fcc6" containerID="6a0ce38bd5ea3db9b8b6b77bb8d6d5944500c722ea11fc48d87dfafbcce88d4b" exitCode=0
May 25 13:21:20.677079 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.676964    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj" event={"ID":"8d209dfa-cd8e-4ecd-a441-002d7832fcc6","Type":"ContainerDied","Data":"6a0ce38bd5ea3db9b8b6b77bb8d6d5944500c722ea11fc48d87dfafbcce88d4b"}
May 25 13:21:20.677079 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:20.676991    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj" event={"ID":"8d209dfa-cd8e-4ecd-a441-002d7832fcc6","Type":"ContainerStarted","Data":"1b663b4fb67f66db475f522e37103521d16aa6920687851b9badd2fb7199f663"}
May 25 13:21:22.688991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:22.688949    2559 generic.go:358] "Generic (PLEG): container finished" podID="626358cc-23d9-41ba-aa03-a04e27ba07e1" containerID="2ebc1dc1b4010ef9fd09368eb01da811c58c1508bb8d1ab92c69324720c93eb5" exitCode=0
May 25 13:21:22.689465 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:22.689028    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4" event={"ID":"626358cc-23d9-41ba-aa03-a04e27ba07e1","Type":"ContainerDied","Data":"2ebc1dc1b4010ef9fd09368eb01da811c58c1508bb8d1ab92c69324720c93eb5"}
May 25 13:21:22.690805 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:22.690659    2559 generic.go:358] "Generic (PLEG): container finished" podID="6b35ba0d-cc5a-4126-82f6-6d58f002f441" containerID="ff6bf5278168f99fb32bc22bdb0e38319304f97ecd2f640c09ea9a5c46570311" exitCode=0
May 25 13:21:22.690805 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:22.690742    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6" event={"ID":"6b35ba0d-cc5a-4126-82f6-6d58f002f441","Type":"ContainerDied","Data":"ff6bf5278168f99fb32bc22bdb0e38319304f97ecd2f640c09ea9a5c46570311"}
May 25 13:21:22.692461 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:22.692440    2559 generic.go:358] "Generic (PLEG): container finished" podID="17f85d51-37ce-46bf-a8f0-6b84ced1a30f" containerID="e66abe898bf6927d19a56d18b49dac4c4c8ab603d1f61792fa63254ef8dbb782" exitCode=0
May 25 13:21:22.692551 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:22.692505    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx" event={"ID":"17f85d51-37ce-46bf-a8f0-6b84ced1a30f","Type":"ContainerDied","Data":"e66abe898bf6927d19a56d18b49dac4c4c8ab603d1f61792fa63254ef8dbb782"}
May 25 13:21:22.694148 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:22.694126    2559 generic.go:358] "Generic (PLEG): container finished" podID="8d209dfa-cd8e-4ecd-a441-002d7832fcc6" containerID="c9a732abeb3dfbbf66c5a17bc8ca1386c4e311844bae2f19ff95c90ca309b6e6" exitCode=0
May 25 13:21:22.694244 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:22.694153    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj" event={"ID":"8d209dfa-cd8e-4ecd-a441-002d7832fcc6","Type":"ContainerDied","Data":"c9a732abeb3dfbbf66c5a17bc8ca1386c4e311844bae2f19ff95c90ca309b6e6"}
May 25 13:21:23.699841 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:23.699808    2559 generic.go:358] "Generic (PLEG): container finished" podID="626358cc-23d9-41ba-aa03-a04e27ba07e1" containerID="563cfe6872b47aa6e2a2cae491838d1cf0a210c553dad920fa44fa3106f376d6" exitCode=0
May 25 13:21:23.700257 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:23.699892    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4" event={"ID":"626358cc-23d9-41ba-aa03-a04e27ba07e1","Type":"ContainerDied","Data":"563cfe6872b47aa6e2a2cae491838d1cf0a210c553dad920fa44fa3106f376d6"}
May 25 13:21:23.701760 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:23.701739    2559 generic.go:358] "Generic (PLEG): container finished" podID="6b35ba0d-cc5a-4126-82f6-6d58f002f441" containerID="ea5a43532bd07b41d5146065e1595739b2e41c51c7f49901b78b0f79857e1e57" exitCode=0
May 25 13:21:23.701875 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:23.701818    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6" event={"ID":"6b35ba0d-cc5a-4126-82f6-6d58f002f441","Type":"ContainerDied","Data":"ea5a43532bd07b41d5146065e1595739b2e41c51c7f49901b78b0f79857e1e57"}
May 25 13:21:23.703552 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:23.703529    2559 generic.go:358] "Generic (PLEG): container finished" podID="17f85d51-37ce-46bf-a8f0-6b84ced1a30f" containerID="7cd537deca703b65623188cae1294a141d7b5dbfc7effe396dbfb7c1eae4edce" exitCode=0
May 25 13:21:23.703657 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:23.703619    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx" event={"ID":"17f85d51-37ce-46bf-a8f0-6b84ced1a30f","Type":"ContainerDied","Data":"7cd537deca703b65623188cae1294a141d7b5dbfc7effe396dbfb7c1eae4edce"}
May 25 13:21:23.705348 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:23.705328    2559 generic.go:358] "Generic (PLEG): container finished" podID="8d209dfa-cd8e-4ecd-a441-002d7832fcc6" containerID="16d0736a41bd4e4fbe068d3517ae26b305344adb200f2adeda72daee7f4c42bc" exitCode=0
May 25 13:21:23.705441 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:23.705385    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj" event={"ID":"8d209dfa-cd8e-4ecd-a441-002d7832fcc6","Type":"ContainerDied","Data":"16d0736a41bd4e4fbe068d3517ae26b305344adb200f2adeda72daee7f4c42bc"}
May 25 13:21:24.840884 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.840863    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"
May 25 13:21:24.882668 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.882644    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"
May 25 13:21:24.909794 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.909770    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"
May 25 13:21:24.916493 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.916471    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"
May 25 13:21:24.988621 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.988544    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fcz4\" (UniqueName: \"kubernetes.io/projected/6b35ba0d-cc5a-4126-82f6-6d58f002f441-kube-api-access-4fcz4\") pod \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\" (UID: \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\") "
May 25 13:21:24.988621 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.988595    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-bundle\") pod \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\" (UID: \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\") "
May 25 13:21:24.988621 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.988618    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b35ba0d-cc5a-4126-82f6-6d58f002f441-bundle\") pod \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\" (UID: \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\") "
May 25 13:21:24.988897 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.988669    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwf8m\" (UniqueName: \"kubernetes.io/projected/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-kube-api-access-mwf8m\") pod \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\" (UID: \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\") "
May 25 13:21:24.988897 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.988694    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-util\") pod \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\" (UID: \"17f85d51-37ce-46bf-a8f0-6b84ced1a30f\") "
May 25 13:21:24.988897 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.988754    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b35ba0d-cc5a-4126-82f6-6d58f002f441-util\") pod \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\" (UID: \"6b35ba0d-cc5a-4126-82f6-6d58f002f441\") "
May 25 13:21:24.989239 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.989211    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b35ba0d-cc5a-4126-82f6-6d58f002f441-bundle" (OuterVolumeSpecName: "bundle") pod "6b35ba0d-cc5a-4126-82f6-6d58f002f441" (UID: "6b35ba0d-cc5a-4126-82f6-6d58f002f441"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:21:24.989396 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.989373    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-bundle" (OuterVolumeSpecName: "bundle") pod "17f85d51-37ce-46bf-a8f0-6b84ced1a30f" (UID: "17f85d51-37ce-46bf-a8f0-6b84ced1a30f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:21:24.991182 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.991151    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-kube-api-access-mwf8m" (OuterVolumeSpecName: "kube-api-access-mwf8m") pod "17f85d51-37ce-46bf-a8f0-6b84ced1a30f" (UID: "17f85d51-37ce-46bf-a8f0-6b84ced1a30f"). InnerVolumeSpecName "kube-api-access-mwf8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:21:24.991284 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.991227    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b35ba0d-cc5a-4126-82f6-6d58f002f441-kube-api-access-4fcz4" (OuterVolumeSpecName: "kube-api-access-4fcz4") pod "6b35ba0d-cc5a-4126-82f6-6d58f002f441" (UID: "6b35ba0d-cc5a-4126-82f6-6d58f002f441"). InnerVolumeSpecName "kube-api-access-4fcz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:21:24.994350 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.994322    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-util" (OuterVolumeSpecName: "util") pod "17f85d51-37ce-46bf-a8f0-6b84ced1a30f" (UID: "17f85d51-37ce-46bf-a8f0-6b84ced1a30f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:21:24.994591 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:24.994569    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b35ba0d-cc5a-4126-82f6-6d58f002f441-util" (OuterVolumeSpecName: "util") pod "6b35ba0d-cc5a-4126-82f6-6d58f002f441" (UID: "6b35ba0d-cc5a-4126-82f6-6d58f002f441"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:21:25.089742 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.089685    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/626358cc-23d9-41ba-aa03-a04e27ba07e1-bundle\") pod \"626358cc-23d9-41ba-aa03-a04e27ba07e1\" (UID: \"626358cc-23d9-41ba-aa03-a04e27ba07e1\") "
May 25 13:21:25.089742 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.089737    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/626358cc-23d9-41ba-aa03-a04e27ba07e1-util\") pod \"626358cc-23d9-41ba-aa03-a04e27ba07e1\" (UID: \"626358cc-23d9-41ba-aa03-a04e27ba07e1\") "
May 25 13:21:25.089953 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.089759    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfxt7\" (UniqueName: \"kubernetes.io/projected/626358cc-23d9-41ba-aa03-a04e27ba07e1-kube-api-access-gfxt7\") pod \"626358cc-23d9-41ba-aa03-a04e27ba07e1\" (UID: \"626358cc-23d9-41ba-aa03-a04e27ba07e1\") "
May 25 13:21:25.089953 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.089808    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-bundle\") pod \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\" (UID: \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\") "
May 25 13:21:25.089953 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.089825    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-util\") pod \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\" (UID: \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\") "
May 25 13:21:25.089953 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.089854    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lmjl\" (UniqueName: \"kubernetes.io/projected/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-kube-api-access-9lmjl\") pod \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\" (UID: \"8d209dfa-cd8e-4ecd-a441-002d7832fcc6\") "
May 25 13:21:25.090143 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.090101    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b35ba0d-cc5a-4126-82f6-6d58f002f441-util\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:21:25.090143 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.090123    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fcz4\" (UniqueName: \"kubernetes.io/projected/6b35ba0d-cc5a-4126-82f6-6d58f002f441-kube-api-access-4fcz4\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:21:25.090143 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.090138    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-bundle\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:21:25.090282 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.090151    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b35ba0d-cc5a-4126-82f6-6d58f002f441-bundle\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:21:25.090282 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.090163    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwf8m\" (UniqueName: \"kubernetes.io/projected/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-kube-api-access-mwf8m\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:21:25.090282 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.090176    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17f85d51-37ce-46bf-a8f0-6b84ced1a30f-util\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:21:25.090407 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.090378    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/626358cc-23d9-41ba-aa03-a04e27ba07e1-bundle" (OuterVolumeSpecName: "bundle") pod "626358cc-23d9-41ba-aa03-a04e27ba07e1" (UID: "626358cc-23d9-41ba-aa03-a04e27ba07e1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:21:25.090590 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.090565    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-bundle" (OuterVolumeSpecName: "bundle") pod "8d209dfa-cd8e-4ecd-a441-002d7832fcc6" (UID: "8d209dfa-cd8e-4ecd-a441-002d7832fcc6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:21:25.092052 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.092032    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-kube-api-access-9lmjl" (OuterVolumeSpecName: "kube-api-access-9lmjl") pod "8d209dfa-cd8e-4ecd-a441-002d7832fcc6" (UID: "8d209dfa-cd8e-4ecd-a441-002d7832fcc6"). InnerVolumeSpecName "kube-api-access-9lmjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:21:25.092301 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.092274    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/626358cc-23d9-41ba-aa03-a04e27ba07e1-kube-api-access-gfxt7" (OuterVolumeSpecName: "kube-api-access-gfxt7") pod "626358cc-23d9-41ba-aa03-a04e27ba07e1" (UID: "626358cc-23d9-41ba-aa03-a04e27ba07e1"). InnerVolumeSpecName "kube-api-access-gfxt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:21:25.095502 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.095466    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/626358cc-23d9-41ba-aa03-a04e27ba07e1-util" (OuterVolumeSpecName: "util") pod "626358cc-23d9-41ba-aa03-a04e27ba07e1" (UID: "626358cc-23d9-41ba-aa03-a04e27ba07e1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:21:25.096205 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.096185    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-util" (OuterVolumeSpecName: "util") pod "8d209dfa-cd8e-4ecd-a441-002d7832fcc6" (UID: "8d209dfa-cd8e-4ecd-a441-002d7832fcc6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:21:25.190973 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.190937    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-bundle\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:21:25.190973 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.190967    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-util\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:21:25.190973 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.190978    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lmjl\" (UniqueName: \"kubernetes.io/projected/8d209dfa-cd8e-4ecd-a441-002d7832fcc6-kube-api-access-9lmjl\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:21:25.191185 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.190989    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/626358cc-23d9-41ba-aa03-a04e27ba07e1-bundle\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:21:25.191185 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.190998    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/626358cc-23d9-41ba-aa03-a04e27ba07e1-util\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:21:25.191185 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.191007    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfxt7\" (UniqueName: \"kubernetes.io/projected/626358cc-23d9-41ba-aa03-a04e27ba07e1-kube-api-access-gfxt7\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:21:25.713438 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.713399    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4" event={"ID":"626358cc-23d9-41ba-aa03-a04e27ba07e1","Type":"ContainerDied","Data":"2a422876a77bf261f493fe206303a096dec657783c12aed3f01259d316433c16"}
May 25 13:21:25.713438 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.713441    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a422876a77bf261f493fe206303a096dec657783c12aed3f01259d316433c16"
May 25 13:21:25.713679 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.713445    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/d54990151e6a3040f48962708f776cca7120e97625eb3a76d6fde2767bbfdb4"
May 25 13:21:25.714989 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.714963    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6" event={"ID":"6b35ba0d-cc5a-4126-82f6-6d58f002f441","Type":"ContainerDied","Data":"7e405432d29d2ba9e21eb55bb32223f53da863418c04ffc2a8b57bba7505ce3a"}
May 25 13:21:25.715114 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.714992    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e405432d29d2ba9e21eb55bb32223f53da863418c04ffc2a8b57bba7505ce3a"
May 25 13:21:25.715114 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.715000    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ef1189e9861fa30b9414ceb420c2d78e85403a7e10097f37afdacfec88vfqb6"
May 25 13:21:25.716660 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.716636    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx" event={"ID":"17f85d51-37ce-46bf-a8f0-6b84ced1a30f","Type":"ContainerDied","Data":"9c62ff10d7e6bfc67a20349e548b9771d29be5df5c2a3b87bc5f16c796a4af87"}
May 25 13:21:25.716660 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.716659    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c62ff10d7e6bfc67a20349e548b9771d29be5df5c2a3b87bc5f16c796a4af87"
May 25 13:21:25.716820 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.716684    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/309c32b5cc4cbd8148882f0ed2adbe6c47ee2761cc0c22c627755a3c30fnxkx"
May 25 13:21:25.718345 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.718321    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj" event={"ID":"8d209dfa-cd8e-4ecd-a441-002d7832fcc6","Type":"ContainerDied","Data":"1b663b4fb67f66db475f522e37103521d16aa6920687851b9badd2fb7199f663"}
May 25 13:21:25.718434 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.718351    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/b432bfd7b8463c35d1aac5aff9103121dc1483a118ad6caa2a17f39e782pbhj"
May 25 13:21:25.718476 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:25.718350    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b663b4fb67f66db475f522e37103521d16aa6920687851b9badd2fb7199f663"
May 25 13:21:34.422130 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422093    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-844548ff4c-k8p9k"]
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422427    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d209dfa-cd8e-4ecd-a441-002d7832fcc6" containerName="extract"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422440    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d209dfa-cd8e-4ecd-a441-002d7832fcc6" containerName="extract"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422449    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b35ba0d-cc5a-4126-82f6-6d58f002f441" containerName="extract"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422455    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b35ba0d-cc5a-4126-82f6-6d58f002f441" containerName="extract"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422462    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17f85d51-37ce-46bf-a8f0-6b84ced1a30f" containerName="util"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422468    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="17f85d51-37ce-46bf-a8f0-6b84ced1a30f" containerName="util"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422476    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17f85d51-37ce-46bf-a8f0-6b84ced1a30f" containerName="pull"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422481    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="17f85d51-37ce-46bf-a8f0-6b84ced1a30f" containerName="pull"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422487    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="626358cc-23d9-41ba-aa03-a04e27ba07e1" containerName="pull"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422492    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="626358cc-23d9-41ba-aa03-a04e27ba07e1" containerName="pull"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422501    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b35ba0d-cc5a-4126-82f6-6d58f002f441" containerName="pull"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422506    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b35ba0d-cc5a-4126-82f6-6d58f002f441" containerName="pull"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422516    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="626358cc-23d9-41ba-aa03-a04e27ba07e1" containerName="util"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422521    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="626358cc-23d9-41ba-aa03-a04e27ba07e1" containerName="util"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422528    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17f85d51-37ce-46bf-a8f0-6b84ced1a30f" containerName="extract"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422532    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="17f85d51-37ce-46bf-a8f0-6b84ced1a30f" containerName="extract"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422540    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d209dfa-cd8e-4ecd-a441-002d7832fcc6" containerName="pull"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422544    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d209dfa-cd8e-4ecd-a441-002d7832fcc6" containerName="pull"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422550    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="626358cc-23d9-41ba-aa03-a04e27ba07e1" containerName="extract"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422554    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="626358cc-23d9-41ba-aa03-a04e27ba07e1" containerName="extract"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422560    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d209dfa-cd8e-4ecd-a441-002d7832fcc6" containerName="util"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422565    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d209dfa-cd8e-4ecd-a441-002d7832fcc6" containerName="util"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422571    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b35ba0d-cc5a-4126-82f6-6d58f002f441" containerName="util"
May 25 13:21:34.422570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422576    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b35ba0d-cc5a-4126-82f6-6d58f002f441" containerName="util"
May 25 13:21:34.423279 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422619    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="8d209dfa-cd8e-4ecd-a441-002d7832fcc6" containerName="extract"
May 25 13:21:34.423279 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422626    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="626358cc-23d9-41ba-aa03-a04e27ba07e1" containerName="extract"
May 25 13:21:34.423279 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422636    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="17f85d51-37ce-46bf-a8f0-6b84ced1a30f" containerName="extract"
May 25 13:21:34.423279 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.422641    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="6b35ba0d-cc5a-4126-82f6-6d58f002f441" containerName="extract"
May 25 13:21:34.432575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.432554    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-k8p9k"
May 25 13:21:34.435515 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.435485    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
May 25 13:21:34.435658 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.435635    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 25 13:21:34.436392 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.436370    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 25 13:21:34.436514 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.436377    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7vd6x\""
May 25 13:21:34.441165 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.441147    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-844548ff4c-k8p9k"]
May 25 13:21:34.466768 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.466741    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg9qk\" (UniqueName: \"kubernetes.io/projected/ee52a3c2-7253-4cb0-96cb-1489027a5c61-kube-api-access-hg9qk\") pod \"dns-operator-controller-manager-844548ff4c-k8p9k\" (UID: \"ee52a3c2-7253-4cb0-96cb-1489027a5c61\") " pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-k8p9k"
May 25 13:21:34.567408 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.567379    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hg9qk\" (UniqueName: \"kubernetes.io/projected/ee52a3c2-7253-4cb0-96cb-1489027a5c61-kube-api-access-hg9qk\") pod \"dns-operator-controller-manager-844548ff4c-k8p9k\" (UID: \"ee52a3c2-7253-4cb0-96cb-1489027a5c61\") " pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-k8p9k"
May 25 13:21:34.577820 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.577785    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg9qk\" (UniqueName: \"kubernetes.io/projected/ee52a3c2-7253-4cb0-96cb-1489027a5c61-kube-api-access-hg9qk\") pod \"dns-operator-controller-manager-844548ff4c-k8p9k\" (UID: \"ee52a3c2-7253-4cb0-96cb-1489027a5c61\") " pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-k8p9k"
May 25 13:21:34.744136 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.744107    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-k8p9k"
May 25 13:21:34.878588 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:34.878563    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-844548ff4c-k8p9k"]
May 25 13:21:34.880600 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:21:34.880569    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee52a3c2_7253_4cb0_96cb_1489027a5c61.slice/crio-f7fb071daac734889b9ca8aaa69d9e01c29927450af6d088cf872464036a25df WatchSource:0}: Error finding container f7fb071daac734889b9ca8aaa69d9e01c29927450af6d088cf872464036a25df: Status 404 returned error can't find the container with id f7fb071daac734889b9ca8aaa69d9e01c29927450af6d088cf872464036a25df
May 25 13:21:35.751457 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:35.751422    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-k8p9k" event={"ID":"ee52a3c2-7253-4cb0-96cb-1489027a5c61","Type":"ContainerStarted","Data":"f7fb071daac734889b9ca8aaa69d9e01c29927450af6d088cf872464036a25df"}
May 25 13:21:37.462362 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:37.462310    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-bqc7m"]
May 25 13:21:37.465546 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:37.465519    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7587b89b76-bqc7m"
May 25 13:21:37.469600 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:37.469581    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-cxp8x\""
May 25 13:21:37.478558 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:37.478532    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-bqc7m"]
May 25 13:21:37.494645 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:37.494616    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9swwm\" (UniqueName: \"kubernetes.io/projected/4f3e4f48-cfe0-405e-8993-de677e493d70-kube-api-access-9swwm\") pod \"authorino-operator-7587b89b76-bqc7m\" (UID: \"4f3e4f48-cfe0-405e-8993-de677e493d70\") " pod="kuadrant-system/authorino-operator-7587b89b76-bqc7m"
May 25 13:21:37.595282 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:37.595253    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9swwm\" (UniqueName: \"kubernetes.io/projected/4f3e4f48-cfe0-405e-8993-de677e493d70-kube-api-access-9swwm\") pod \"authorino-operator-7587b89b76-bqc7m\" (UID: \"4f3e4f48-cfe0-405e-8993-de677e493d70\") " pod="kuadrant-system/authorino-operator-7587b89b76-bqc7m"
May 25 13:21:37.604253 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:37.604221    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9swwm\" (UniqueName: \"kubernetes.io/projected/4f3e4f48-cfe0-405e-8993-de677e493d70-kube-api-access-9swwm\") pod \"authorino-operator-7587b89b76-bqc7m\" (UID: \"4f3e4f48-cfe0-405e-8993-de677e493d70\") " pod="kuadrant-system/authorino-operator-7587b89b76-bqc7m"
May 25 13:21:37.759143 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:37.759105    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-k8p9k" event={"ID":"ee52a3c2-7253-4cb0-96cb-1489027a5c61","Type":"ContainerStarted","Data":"bcc6299fb05524efc33860e6efa6fe03601f1faf920cca0910773813fd7b37d6"}
May 25 13:21:37.759308 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:37.759183    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-k8p9k"
May 25 13:21:37.776001 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:37.775977    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7587b89b76-bqc7m"
May 25 13:21:37.784261 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:37.784218    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-k8p9k" podStartSLOduration=1.163368067 podStartE2EDuration="3.784203318s" podCreationTimestamp="2026-05-25 13:21:34 +0000 UTC" firstStartedPulling="2026-05-25 13:21:34.88277043 +0000 UTC m=+961.601733547" lastFinishedPulling="2026-05-25 13:21:37.503605679 +0000 UTC m=+964.222568798" observedRunningTime="2026-05-25 13:21:37.781039499 +0000 UTC m=+964.500002640" watchObservedRunningTime="2026-05-25 13:21:37.784203318 +0000 UTC m=+964.503166456"
May 25 13:21:37.924618 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:37.924585    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7587b89b76-bqc7m"]
May 25 13:21:37.933883 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:21:37.933855    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f3e4f48_cfe0_405e_8993_de677e493d70.slice/crio-9a2a4ba801c4fe75bd7d1051981dd43ef3618aa2d6f165e2f2a7e761ac6530a6 WatchSource:0}: Error finding container 9a2a4ba801c4fe75bd7d1051981dd43ef3618aa2d6f165e2f2a7e761ac6530a6: Status 404 returned error can't find the container with id 9a2a4ba801c4fe75bd7d1051981dd43ef3618aa2d6f165e2f2a7e761ac6530a6
May 25 13:21:38.765657 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:38.765617    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7587b89b76-bqc7m" event={"ID":"4f3e4f48-cfe0-405e-8993-de677e493d70","Type":"ContainerStarted","Data":"9a2a4ba801c4fe75bd7d1051981dd43ef3618aa2d6f165e2f2a7e761ac6530a6"}
May 25 13:21:39.769876 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:39.769817    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7587b89b76-bqc7m" event={"ID":"4f3e4f48-cfe0-405e-8993-de677e493d70","Type":"ContainerStarted","Data":"7d2f8b97e78dfd2457f667325c00e76bd1a2188ea70af35f0d53271c3017966b"}
May 25 13:21:39.770249 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:39.769993    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7587b89b76-bqc7m"
May 25 13:21:39.806305 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:39.806255    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7587b89b76-bqc7m" podStartSLOduration=1.4219991539999999 podStartE2EDuration="2.806240477s" podCreationTimestamp="2026-05-25 13:21:37 +0000 UTC" firstStartedPulling="2026-05-25 13:21:37.935960179 +0000 UTC m=+964.654923296" lastFinishedPulling="2026-05-25 13:21:39.3202015 +0000 UTC m=+966.039164619" observedRunningTime="2026-05-25 13:21:39.805602848 +0000 UTC m=+966.524565985" watchObservedRunningTime="2026-05-25 13:21:39.806240477 +0000 UTC m=+966.525203615"
May 25 13:21:41.027010 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:41.026975    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc"]
May 25 13:21:41.031855 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:41.031838    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc"
May 25 13:21:41.035006 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:41.034985    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bvbh7\""
May 25 13:21:41.056626 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:41.056591    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc"]
May 25 13:21:41.128380 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:41.128345    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwj8d\" (UniqueName: \"kubernetes.io/projected/4ce8a068-b18e-41f8-8175-70487ebafe61-kube-api-access-kwj8d\") pod \"kuadrant-operator-controller-manager-57dd85bb4-rrgxc\" (UID: \"4ce8a068-b18e-41f8-8175-70487ebafe61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc"
May 25 13:21:41.128547 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:41.128385    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ce8a068-b18e-41f8-8175-70487ebafe61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57dd85bb4-rrgxc\" (UID: \"4ce8a068-b18e-41f8-8175-70487ebafe61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc"
May 25 13:21:41.229600 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:41.229556    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kwj8d\" (UniqueName: \"kubernetes.io/projected/4ce8a068-b18e-41f8-8175-70487ebafe61-kube-api-access-kwj8d\") pod \"kuadrant-operator-controller-manager-57dd85bb4-rrgxc\" (UID: \"4ce8a068-b18e-41f8-8175-70487ebafe61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc"
May 25 13:21:41.229600 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:41.229606    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ce8a068-b18e-41f8-8175-70487ebafe61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57dd85bb4-rrgxc\" (UID: \"4ce8a068-b18e-41f8-8175-70487ebafe61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc"
May 25 13:21:41.230046 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:41.230025    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ce8a068-b18e-41f8-8175-70487ebafe61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57dd85bb4-rrgxc\" (UID: \"4ce8a068-b18e-41f8-8175-70487ebafe61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc"
May 25 13:21:41.239132 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:41.239113    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwj8d\" (UniqueName: \"kubernetes.io/projected/4ce8a068-b18e-41f8-8175-70487ebafe61-kube-api-access-kwj8d\") pod \"kuadrant-operator-controller-manager-57dd85bb4-rrgxc\" (UID: \"4ce8a068-b18e-41f8-8175-70487ebafe61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc"
May 25 13:21:41.342380 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:41.342276    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc"
May 25 13:21:41.498465 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:41.498432    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc"]
May 25 13:21:41.501366 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:21:41.501339    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ce8a068_b18e_41f8_8175_70487ebafe61.slice/crio-98b7e5f8c3e4f0c8edc8c473ba65f96c2f90b1d26914aca3a8f8c62065f94ad5 WatchSource:0}: Error finding container 98b7e5f8c3e4f0c8edc8c473ba65f96c2f90b1d26914aca3a8f8c62065f94ad5: Status 404 returned error can't find the container with id 98b7e5f8c3e4f0c8edc8c473ba65f96c2f90b1d26914aca3a8f8c62065f94ad5
May 25 13:21:41.778422 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:41.778386    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc" event={"ID":"4ce8a068-b18e-41f8-8175-70487ebafe61","Type":"ContainerStarted","Data":"98b7e5f8c3e4f0c8edc8c473ba65f96c2f90b1d26914aca3a8f8c62065f94ad5"}
May 25 13:21:47.805355 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:47.805319    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc" event={"ID":"4ce8a068-b18e-41f8-8175-70487ebafe61","Type":"ContainerStarted","Data":"d2f6422d7d0f0e86e2226d8fe088d23e8ee62279f4fd2deb410ea955b631a1d5"}
May 25 13:21:47.805755 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:47.805488    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc"
May 25 13:21:47.938597 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:47.938545    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc" podStartSLOduration=1.743104309 podStartE2EDuration="7.938531538s" podCreationTimestamp="2026-05-25 13:21:40 +0000 UTC" firstStartedPulling="2026-05-25 13:21:41.504225535 +0000 UTC m=+968.223188652" lastFinishedPulling="2026-05-25 13:21:47.699652747 +0000 UTC m=+974.418615881" observedRunningTime="2026-05-25 13:21:47.930262527 +0000 UTC m=+974.649225666" watchObservedRunningTime="2026-05-25 13:21:47.938531538 +0000 UTC m=+974.657494676"
May 25 13:21:48.768149 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:48.768122    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-844548ff4c-k8p9k"
May 25 13:21:50.775567 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:50.775537    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7587b89b76-bqc7m"
May 25 13:21:58.811944 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:21:58.811913    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57dd85bb4-rrgxc"
May 25 13:22:35.496145 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:35.496109    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79cbc94b89-qrhp7"]
May 25 13:22:35.498449 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:35.498432    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-qrhp7"
May 25 13:22:35.500611 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:35.500590    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-btmv6\""
May 25 13:22:35.504052 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:35.504015    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-qrhp7"]
May 25 13:22:35.595872 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:35.595829    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9888\" (UniqueName: \"kubernetes.io/projected/b562b4ab-dc1c-485e-bfdc-abcba44d49e1-kube-api-access-q9888\") pod \"authorino-79cbc94b89-qrhp7\" (UID: \"b562b4ab-dc1c-485e-bfdc-abcba44d49e1\") " pod="kuadrant-system/authorino-79cbc94b89-qrhp7"
May 25 13:22:35.696831 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:35.696793    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q9888\" (UniqueName: \"kubernetes.io/projected/b562b4ab-dc1c-485e-bfdc-abcba44d49e1-kube-api-access-q9888\") pod \"authorino-79cbc94b89-qrhp7\" (UID: \"b562b4ab-dc1c-485e-bfdc-abcba44d49e1\") " pod="kuadrant-system/authorino-79cbc94b89-qrhp7"
May 25 13:22:35.705267 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:35.705243    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9888\" (UniqueName: \"kubernetes.io/projected/b562b4ab-dc1c-485e-bfdc-abcba44d49e1-kube-api-access-q9888\") pod \"authorino-79cbc94b89-qrhp7\" (UID: \"b562b4ab-dc1c-485e-bfdc-abcba44d49e1\") " pod="kuadrant-system/authorino-79cbc94b89-qrhp7"
May 25 13:22:35.808473 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:35.808392    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-qrhp7"
May 25 13:22:35.934168 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:35.934132    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-qrhp7"]
May 25 13:22:35.935334 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:22:35.935306    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb562b4ab_dc1c_485e_bfdc_abcba44d49e1.slice/crio-b1557aacb682864d4c1329659ce369bd3349b2d9a97fec5c9a48687f6463eaaf WatchSource:0}: Error finding container b1557aacb682864d4c1329659ce369bd3349b2d9a97fec5c9a48687f6463eaaf: Status 404 returned error can't find the container with id b1557aacb682864d4c1329659ce369bd3349b2d9a97fec5c9a48687f6463eaaf
May 25 13:22:35.980032 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:35.980002    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-qrhp7" event={"ID":"b562b4ab-dc1c-485e-bfdc-abcba44d49e1","Type":"ContainerStarted","Data":"b1557aacb682864d4c1329659ce369bd3349b2d9a97fec5c9a48687f6463eaaf"}
May 25 13:22:37.988829 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:37.988747    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-qrhp7" event={"ID":"b562b4ab-dc1c-485e-bfdc-abcba44d49e1","Type":"ContainerStarted","Data":"398583bc7241525283b5f55226c18c7618c5e14d95d3c50bd6af833fbe9b1eb3"}
May 25 13:22:38.007085 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:38.007019    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-79cbc94b89-qrhp7" podStartSLOduration=1.375242843 podStartE2EDuration="3.007001247s" podCreationTimestamp="2026-05-25 13:22:35 +0000 UTC" firstStartedPulling="2026-05-25 13:22:35.936801805 +0000 UTC m=+1022.655764934" lastFinishedPulling="2026-05-25 13:22:37.568560207 +0000 UTC m=+1024.287523338" observedRunningTime="2026-05-25 13:22:38.002776874 +0000 UTC m=+1024.721740013" watchObservedRunningTime="2026-05-25 13:22:38.007001247 +0000 UTC m=+1024.725964388"
May 25 13:22:57.592821 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:57.592780    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-qrhp7"]
May 25 13:22:57.593200 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:57.592971    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-79cbc94b89-qrhp7" podUID="b562b4ab-dc1c-485e-bfdc-abcba44d49e1" containerName="authorino" containerID="cri-o://398583bc7241525283b5f55226c18c7618c5e14d95d3c50bd6af833fbe9b1eb3" gracePeriod=30
May 25 13:22:57.838536 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:57.838513    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-qrhp7"
May 25 13:22:57.987937 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:57.987856    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9888\" (UniqueName: \"kubernetes.io/projected/b562b4ab-dc1c-485e-bfdc-abcba44d49e1-kube-api-access-q9888\") pod \"b562b4ab-dc1c-485e-bfdc-abcba44d49e1\" (UID: \"b562b4ab-dc1c-485e-bfdc-abcba44d49e1\") "
May 25 13:22:57.990084 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:57.990051    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b562b4ab-dc1c-485e-bfdc-abcba44d49e1-kube-api-access-q9888" (OuterVolumeSpecName: "kube-api-access-q9888") pod "b562b4ab-dc1c-485e-bfdc-abcba44d49e1" (UID: "b562b4ab-dc1c-485e-bfdc-abcba44d49e1"). InnerVolumeSpecName "kube-api-access-q9888". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:22:58.060162 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:58.060126    2559 generic.go:358] "Generic (PLEG): container finished" podID="b562b4ab-dc1c-485e-bfdc-abcba44d49e1" containerID="398583bc7241525283b5f55226c18c7618c5e14d95d3c50bd6af833fbe9b1eb3" exitCode=0
May 25 13:22:58.060343 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:58.060168    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-qrhp7" event={"ID":"b562b4ab-dc1c-485e-bfdc-abcba44d49e1","Type":"ContainerDied","Data":"398583bc7241525283b5f55226c18c7618c5e14d95d3c50bd6af833fbe9b1eb3"}
May 25 13:22:58.060343 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:58.060178    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79cbc94b89-qrhp7"
May 25 13:22:58.060343 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:58.060195    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79cbc94b89-qrhp7" event={"ID":"b562b4ab-dc1c-485e-bfdc-abcba44d49e1","Type":"ContainerDied","Data":"b1557aacb682864d4c1329659ce369bd3349b2d9a97fec5c9a48687f6463eaaf"}
May 25 13:22:58.060343 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:58.060217    2559 scope.go:117] "RemoveContainer" containerID="398583bc7241525283b5f55226c18c7618c5e14d95d3c50bd6af833fbe9b1eb3"
May 25 13:22:58.068876 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:58.068854    2559 scope.go:117] "RemoveContainer" containerID="398583bc7241525283b5f55226c18c7618c5e14d95d3c50bd6af833fbe9b1eb3"
May 25 13:22:58.069155 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:22:58.069135    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"398583bc7241525283b5f55226c18c7618c5e14d95d3c50bd6af833fbe9b1eb3\": container with ID starting with 398583bc7241525283b5f55226c18c7618c5e14d95d3c50bd6af833fbe9b1eb3 not found: ID does not exist" containerID="398583bc7241525283b5f55226c18c7618c5e14d95d3c50bd6af833fbe9b1eb3"
May 25 13:22:58.069195 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:58.069165    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"398583bc7241525283b5f55226c18c7618c5e14d95d3c50bd6af833fbe9b1eb3"} err="failed to get container status \"398583bc7241525283b5f55226c18c7618c5e14d95d3c50bd6af833fbe9b1eb3\": rpc error: code = NotFound desc = could not find container \"398583bc7241525283b5f55226c18c7618c5e14d95d3c50bd6af833fbe9b1eb3\": container with ID starting with 398583bc7241525283b5f55226c18c7618c5e14d95d3c50bd6af833fbe9b1eb3 not found: ID does not exist"
May 25 13:22:58.080136 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:58.080062    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-qrhp7"]
May 25 13:22:58.082173 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:58.082150    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-79cbc94b89-qrhp7"]
May 25 13:22:58.089232 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:58.089212    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9888\" (UniqueName: \"kubernetes.io/projected/b562b4ab-dc1c-485e-bfdc-abcba44d49e1-kube-api-access-q9888\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:22:59.869162 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:22:59.869119    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b562b4ab-dc1c-485e-bfdc-abcba44d49e1" path="/var/lib/kubelet/pods/b562b4ab-dc1c-485e-bfdc-abcba44d49e1/volumes"
May 25 13:23:06.560581 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.560545    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"]
May 25 13:23:06.561007 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.560903    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b562b4ab-dc1c-485e-bfdc-abcba44d49e1" containerName="authorino"
May 25 13:23:06.561007 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.560914    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b562b4ab-dc1c-485e-bfdc-abcba44d49e1" containerName="authorino"
May 25 13:23:06.561007 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.560967    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="b562b4ab-dc1c-485e-bfdc-abcba44d49e1" containerName="authorino"
May 25 13:23:06.565432 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.565408    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.579708 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.579675    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"]
May 25 13:23:06.663023 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.662980    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/598289c4-26ea-49fc-8e24-c662d5be93d8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.663280 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.663066    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/598289c4-26ea-49fc-8e24-c662d5be93d8-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.663280 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.663101    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/598289c4-26ea-49fc-8e24-c662d5be93d8-local-certs\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.663280 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.663123    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/598289c4-26ea-49fc-8e24-c662d5be93d8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.663280 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.663155    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/598289c4-26ea-49fc-8e24-c662d5be93d8-cacerts\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.663280 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.663181    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrbk5\" (UniqueName: \"kubernetes.io/projected/598289c4-26ea-49fc-8e24-c662d5be93d8-kube-api-access-vrbk5\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.663280 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.663229    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/598289c4-26ea-49fc-8e24-c662d5be93d8-istio-token\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.764321 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.764285    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/598289c4-26ea-49fc-8e24-c662d5be93d8-istio-token\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.764501 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.764345    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/598289c4-26ea-49fc-8e24-c662d5be93d8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.764501 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.764376    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/598289c4-26ea-49fc-8e24-c662d5be93d8-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.764501 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.764400    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/598289c4-26ea-49fc-8e24-c662d5be93d8-local-certs\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.764501 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.764415    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/598289c4-26ea-49fc-8e24-c662d5be93d8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.764501 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.764443    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/598289c4-26ea-49fc-8e24-c662d5be93d8-cacerts\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.764501 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.764467    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vrbk5\" (UniqueName: \"kubernetes.io/projected/598289c4-26ea-49fc-8e24-c662d5be93d8-kube-api-access-vrbk5\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.765326 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.765294    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/598289c4-26ea-49fc-8e24-c662d5be93d8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.766921 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.766890    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/598289c4-26ea-49fc-8e24-c662d5be93d8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.767010 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.766930    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/598289c4-26ea-49fc-8e24-c662d5be93d8-local-certs\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.767049 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.767009    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/598289c4-26ea-49fc-8e24-c662d5be93d8-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.767123 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.767100    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/598289c4-26ea-49fc-8e24-c662d5be93d8-cacerts\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.777035 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.777006    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/598289c4-26ea-49fc-8e24-c662d5be93d8-istio-token\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.778152 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.778131    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrbk5\" (UniqueName: \"kubernetes.io/projected/598289c4-26ea-49fc-8e24-c662d5be93d8-kube-api-access-vrbk5\") pod \"istiod-openshift-gateway-798958bb55-6p94s\" (UID: \"598289c4-26ea-49fc-8e24-c662d5be93d8\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:06.875035 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:06.874952    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:07.039892 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:07.039859    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"]
May 25 13:23:07.040433 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:23:07.040406    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598289c4_26ea_49fc_8e24_c662d5be93d8.slice/crio-1cf10a246ee49f8ed59e4b53812488cb2dde740a906216c2919d68bc251d12f1 WatchSource:0}: Error finding container 1cf10a246ee49f8ed59e4b53812488cb2dde740a906216c2919d68bc251d12f1: Status 404 returned error can't find the container with id 1cf10a246ee49f8ed59e4b53812488cb2dde740a906216c2919d68bc251d12f1
May 25 13:23:07.042513 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:07.042480    2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 13:23:07.042575 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:07.042557    2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 13:23:07.099025 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:07.098994    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s" event={"ID":"598289c4-26ea-49fc-8e24-c662d5be93d8","Type":"ContainerStarted","Data":"1cf10a246ee49f8ed59e4b53812488cb2dde740a906216c2919d68bc251d12f1"}
May 25 13:23:08.108027 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.107988    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s" event={"ID":"598289c4-26ea-49fc-8e24-c662d5be93d8","Type":"ContainerStarted","Data":"112dc72fe72be3339bb150290602c07dcf015c4cfc22ebb591077db80900f206"}
May 25 13:23:08.108471 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.108177    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:08.110130 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.110105    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s"
May 25 13:23:08.136948 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.136365    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-6p94s" podStartSLOduration=2.13634511 podStartE2EDuration="2.13634511s" podCreationTimestamp="2026-05-25 13:23:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:23:08.132087203 +0000 UTC m=+1054.851050342" watchObservedRunningTime="2026-05-25 13:23:08.13634511 +0000 UTC m=+1054.855308249"
May 25 13:23:08.203365 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.203316    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"]
May 25 13:23:08.203597 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.203574    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z" podUID="b95365e2-13be-4cf6-ae39-b13c23078710" containerName="discovery" containerID="cri-o://44931a755fc4dc0abe07d26f5cf06044491f84094c43d24761d6ac2a234783e7" gracePeriod=30
May 25 13:23:08.490680 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.490653    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:23:08.579049 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.579004    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b95365e2-13be-4cf6-ae39-b13c23078710-istio-csr-ca-configmap\") pod \"b95365e2-13be-4cf6-ae39-b13c23078710\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") "
May 25 13:23:08.579232 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.579059    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b95365e2-13be-4cf6-ae39-b13c23078710-istio-token\") pod \"b95365e2-13be-4cf6-ae39-b13c23078710\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") "
May 25 13:23:08.579232 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.579118    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-istio-kubeconfig\") pod \"b95365e2-13be-4cf6-ae39-b13c23078710\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") "
May 25 13:23:08.579232 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.579155    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cs5r\" (UniqueName: \"kubernetes.io/projected/b95365e2-13be-4cf6-ae39-b13c23078710-kube-api-access-2cs5r\") pod \"b95365e2-13be-4cf6-ae39-b13c23078710\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") "
May 25 13:23:08.579232 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.579183    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b95365e2-13be-4cf6-ae39-b13c23078710-local-certs\") pod \"b95365e2-13be-4cf6-ae39-b13c23078710\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") "
May 25 13:23:08.579232 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.579208    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-istio-csr-dns-cert\") pod \"b95365e2-13be-4cf6-ae39-b13c23078710\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") "
May 25 13:23:08.579232 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.579235    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-cacerts\") pod \"b95365e2-13be-4cf6-ae39-b13c23078710\" (UID: \"b95365e2-13be-4cf6-ae39-b13c23078710\") "
May 25 13:23:08.579523 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.579498    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b95365e2-13be-4cf6-ae39-b13c23078710-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b95365e2-13be-4cf6-ae39-b13c23078710" (UID: "b95365e2-13be-4cf6-ae39-b13c23078710"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 13:23:08.581902 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.581867    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b95365e2-13be-4cf6-ae39-b13c23078710-kube-api-access-2cs5r" (OuterVolumeSpecName: "kube-api-access-2cs5r") pod "b95365e2-13be-4cf6-ae39-b13c23078710" (UID: "b95365e2-13be-4cf6-ae39-b13c23078710"). InnerVolumeSpecName "kube-api-access-2cs5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:23:08.582046 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.581931    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b95365e2-13be-4cf6-ae39-b13c23078710" (UID: "b95365e2-13be-4cf6-ae39-b13c23078710"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:23:08.582115 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.582053    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b95365e2-13be-4cf6-ae39-b13c23078710-istio-token" (OuterVolumeSpecName: "istio-token") pod "b95365e2-13be-4cf6-ae39-b13c23078710" (UID: "b95365e2-13be-4cf6-ae39-b13c23078710"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:23:08.582115 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.582087    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b95365e2-13be-4cf6-ae39-b13c23078710-local-certs" (OuterVolumeSpecName: "local-certs") pod "b95365e2-13be-4cf6-ae39-b13c23078710" (UID: "b95365e2-13be-4cf6-ae39-b13c23078710"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:23:08.582214 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.582167    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-cacerts" (OuterVolumeSpecName: "cacerts") pod "b95365e2-13be-4cf6-ae39-b13c23078710" (UID: "b95365e2-13be-4cf6-ae39-b13c23078710"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:23:08.582266 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.582250    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b95365e2-13be-4cf6-ae39-b13c23078710" (UID: "b95365e2-13be-4cf6-ae39-b13c23078710"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:23:08.680139 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.680045    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cs5r\" (UniqueName: \"kubernetes.io/projected/b95365e2-13be-4cf6-ae39-b13c23078710-kube-api-access-2cs5r\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:23:08.680139 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.680078    2559 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b95365e2-13be-4cf6-ae39-b13c23078710-local-certs\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:23:08.680139 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.680091    2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-istio-csr-dns-cert\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:23:08.680139 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.680104    2559 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-cacerts\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:23:08.680139 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.680116    2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b95365e2-13be-4cf6-ae39-b13c23078710-istio-csr-ca-configmap\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:23:08.680139 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.680129    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b95365e2-13be-4cf6-ae39-b13c23078710-istio-token\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:23:08.680139 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:08.680143    2559 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b95365e2-13be-4cf6-ae39-b13c23078710-istio-kubeconfig\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:23:09.113890 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:09.113851    2559 generic.go:358] "Generic (PLEG): container finished" podID="b95365e2-13be-4cf6-ae39-b13c23078710" containerID="44931a755fc4dc0abe07d26f5cf06044491f84094c43d24761d6ac2a234783e7" exitCode=0
May 25 13:23:09.114293 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:09.113917    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"
May 25 13:23:09.114293 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:09.113933    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z" event={"ID":"b95365e2-13be-4cf6-ae39-b13c23078710","Type":"ContainerDied","Data":"44931a755fc4dc0abe07d26f5cf06044491f84094c43d24761d6ac2a234783e7"}
May 25 13:23:09.114293 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:09.113968    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z" event={"ID":"b95365e2-13be-4cf6-ae39-b13c23078710","Type":"ContainerDied","Data":"0c55f9932e5fbefef933a059cd2425e3270157fbfb4554e4a63eb85d2d7181a5"}
May 25 13:23:09.114293 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:09.113984    2559 scope.go:117] "RemoveContainer" containerID="44931a755fc4dc0abe07d26f5cf06044491f84094c43d24761d6ac2a234783e7"
May 25 13:23:09.122749 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:09.122702    2559 scope.go:117] "RemoveContainer" containerID="44931a755fc4dc0abe07d26f5cf06044491f84094c43d24761d6ac2a234783e7"
May 25 13:23:09.122982 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:23:09.122963    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44931a755fc4dc0abe07d26f5cf06044491f84094c43d24761d6ac2a234783e7\": container with ID starting with 44931a755fc4dc0abe07d26f5cf06044491f84094c43d24761d6ac2a234783e7 not found: ID does not exist" containerID="44931a755fc4dc0abe07d26f5cf06044491f84094c43d24761d6ac2a234783e7"
May 25 13:23:09.123037 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:09.122992    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44931a755fc4dc0abe07d26f5cf06044491f84094c43d24761d6ac2a234783e7"} err="failed to get container status \"44931a755fc4dc0abe07d26f5cf06044491f84094c43d24761d6ac2a234783e7\": rpc error: code = NotFound desc = could not find container \"44931a755fc4dc0abe07d26f5cf06044491f84094c43d24761d6ac2a234783e7\": container with ID starting with 44931a755fc4dc0abe07d26f5cf06044491f84094c43d24761d6ac2a234783e7 not found: ID does not exist"
May 25 13:23:09.161972 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:09.161935    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"]
May 25 13:23:09.167945 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:09.167916    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-767cf5cf55-hn88z"]
May 25 13:23:09.869750 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:09.869702    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b95365e2-13be-4cf6-ae39-b13c23078710" path="/var/lib/kubelet/pods/b95365e2-13be-4cf6-ae39-b13c23078710/volumes"
May 25 13:23:16.115332 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.115248    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-79f457656b-s4xkh"]
May 25 13:23:16.115698 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.115578    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b95365e2-13be-4cf6-ae39-b13c23078710" containerName="discovery"
May 25 13:23:16.115698 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.115589    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b95365e2-13be-4cf6-ae39-b13c23078710" containerName="discovery"
May 25 13:23:16.115698 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.115655    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="b95365e2-13be-4cf6-ae39-b13c23078710" containerName="discovery"
May 25 13:23:16.121002 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.120982    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-79f457656b-s4xkh"
May 25 13:23:16.123290 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.123264    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
May 25 13:23:16.123475 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.123347    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
May 25 13:23:16.123475 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.123421    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-bhxnf\""
May 25 13:23:16.124511 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.124381    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
May 25 13:23:16.127767 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.127742    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-79f457656b-s4xkh"]
May 25 13:23:16.139829 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.139800    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xcnm\" (UniqueName: \"kubernetes.io/projected/d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0-kube-api-access-2xcnm\") pod \"seaweedfs-79f457656b-s4xkh\" (UID: \"d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0\") " pod="kserve/seaweedfs-79f457656b-s4xkh"
May 25 13:23:16.139989 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.139855    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0-data\") pod \"seaweedfs-79f457656b-s4xkh\" (UID: \"d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0\") " pod="kserve/seaweedfs-79f457656b-s4xkh"
May 25 13:23:16.240873 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.240836    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2xcnm\" (UniqueName: \"kubernetes.io/projected/d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0-kube-api-access-2xcnm\") pod \"seaweedfs-79f457656b-s4xkh\" (UID: \"d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0\") " pod="kserve/seaweedfs-79f457656b-s4xkh"
May 25 13:23:16.240873 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.240888    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0-data\") pod \"seaweedfs-79f457656b-s4xkh\" (UID: \"d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0\") " pod="kserve/seaweedfs-79f457656b-s4xkh"
May 25 13:23:16.241265 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.241249    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0-data\") pod \"seaweedfs-79f457656b-s4xkh\" (UID: \"d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0\") " pod="kserve/seaweedfs-79f457656b-s4xkh"
May 25 13:23:16.252671 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.252642    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xcnm\" (UniqueName: \"kubernetes.io/projected/d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0-kube-api-access-2xcnm\") pod \"seaweedfs-79f457656b-s4xkh\" (UID: \"d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0\") " pod="kserve/seaweedfs-79f457656b-s4xkh"
May 25 13:23:16.431790 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.431696    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-79f457656b-s4xkh"
May 25 13:23:16.568313 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:16.568288    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-79f457656b-s4xkh"]
May 25 13:23:16.570540 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:23:16.570509    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd74918be_6d2c_4d4d_8d8f_6c05b2c2b7b0.slice/crio-cb142daabd578d91ccf7cd58b4944ea697cd5b524121b41d0d33218ae8d592da WatchSource:0}: Error finding container cb142daabd578d91ccf7cd58b4944ea697cd5b524121b41d0d33218ae8d592da: Status 404 returned error can't find the container with id cb142daabd578d91ccf7cd58b4944ea697cd5b524121b41d0d33218ae8d592da
May 25 13:23:17.149493 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:17.149456    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-79f457656b-s4xkh" event={"ID":"d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0","Type":"ContainerStarted","Data":"cb142daabd578d91ccf7cd58b4944ea697cd5b524121b41d0d33218ae8d592da"}
May 25 13:23:20.161841 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:20.161805    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-79f457656b-s4xkh" event={"ID":"d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0","Type":"ContainerStarted","Data":"163f5e24c9a8889965ab2eabb8fdcc1b6ef1467373c950b5b49daea208c1c46e"}
May 25 13:23:20.162228 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:20.161916    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-79f457656b-s4xkh"
May 25 13:23:20.180432 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:20.180381    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/seaweedfs-79f457656b-s4xkh" podStartSLOduration=1.139240988 podStartE2EDuration="4.180365532s" podCreationTimestamp="2026-05-25 13:23:16 +0000 UTC" firstStartedPulling="2026-05-25 13:23:16.571800774 +0000 UTC m=+1063.290763891" lastFinishedPulling="2026-05-25 13:23:19.612925304 +0000 UTC m=+1066.331888435" observedRunningTime="2026-05-25 13:23:20.178257754 +0000 UTC m=+1066.897220895" watchObservedRunningTime="2026-05-25 13:23:20.180365532 +0000 UTC m=+1066.899328670"
May 25 13:23:26.167521 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:23:26.167488    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-79f457656b-s4xkh"
May 25 13:24:26.698191 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:26.698152    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-696fc77849-9ntd8"]
May 25 13:24:26.701560 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:26.701542    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-9ntd8"
May 25 13:24:26.703957 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:26.703934    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
May 25 13:24:26.704095 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:26.703939    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-f88dj\""
May 25 13:24:26.711441 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:26.711405    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-9ntd8"]
May 25 13:24:26.819544 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:26.819508    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd43edef-f491-4b47-ab13-002d03a58f00-cert\") pod \"odh-model-controller-696fc77849-9ntd8\" (UID: \"fd43edef-f491-4b47-ab13-002d03a58f00\") " pod="kserve/odh-model-controller-696fc77849-9ntd8"
May 25 13:24:26.819744 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:26.819625    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frw9n\" (UniqueName: \"kubernetes.io/projected/fd43edef-f491-4b47-ab13-002d03a58f00-kube-api-access-frw9n\") pod \"odh-model-controller-696fc77849-9ntd8\" (UID: \"fd43edef-f491-4b47-ab13-002d03a58f00\") " pod="kserve/odh-model-controller-696fc77849-9ntd8"
May 25 13:24:26.920686 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:26.920651    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd43edef-f491-4b47-ab13-002d03a58f00-cert\") pod \"odh-model-controller-696fc77849-9ntd8\" (UID: \"fd43edef-f491-4b47-ab13-002d03a58f00\") " pod="kserve/odh-model-controller-696fc77849-9ntd8"
May 25 13:24:26.920920 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:26.920857    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-frw9n\" (UniqueName: \"kubernetes.io/projected/fd43edef-f491-4b47-ab13-002d03a58f00-kube-api-access-frw9n\") pod \"odh-model-controller-696fc77849-9ntd8\" (UID: \"fd43edef-f491-4b47-ab13-002d03a58f00\") " pod="kserve/odh-model-controller-696fc77849-9ntd8"
May 25 13:24:26.923181 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:26.923147    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd43edef-f491-4b47-ab13-002d03a58f00-cert\") pod \"odh-model-controller-696fc77849-9ntd8\" (UID: \"fd43edef-f491-4b47-ab13-002d03a58f00\") " pod="kserve/odh-model-controller-696fc77849-9ntd8"
May 25 13:24:26.929460 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:26.929434    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frw9n\" (UniqueName: \"kubernetes.io/projected/fd43edef-f491-4b47-ab13-002d03a58f00-kube-api-access-frw9n\") pod \"odh-model-controller-696fc77849-9ntd8\" (UID: \"fd43edef-f491-4b47-ab13-002d03a58f00\") " pod="kserve/odh-model-controller-696fc77849-9ntd8"
May 25 13:24:27.015033 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:27.014998    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-9ntd8"
May 25 13:24:27.150552 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:27.150527    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-9ntd8"]
May 25 13:24:27.152843 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:24:27.152813    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd43edef_f491_4b47_ab13_002d03a58f00.slice/crio-d2f47e9b8ab61dc125c0e81fbfdb02ea5aadc1bbb1ebeee8c1cf928d8d0b42bd WatchSource:0}: Error finding container d2f47e9b8ab61dc125c0e81fbfdb02ea5aadc1bbb1ebeee8c1cf928d8d0b42bd: Status 404 returned error can't find the container with id d2f47e9b8ab61dc125c0e81fbfdb02ea5aadc1bbb1ebeee8c1cf928d8d0b42bd
May 25 13:24:27.154442 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:27.154426    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 13:24:27.400672 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:27.400587    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-9ntd8" event={"ID":"fd43edef-f491-4b47-ab13-002d03a58f00","Type":"ContainerStarted","Data":"d2f47e9b8ab61dc125c0e81fbfdb02ea5aadc1bbb1ebeee8c1cf928d8d0b42bd"}
May 25 13:24:30.413921 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:30.413882    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-9ntd8" event={"ID":"fd43edef-f491-4b47-ab13-002d03a58f00","Type":"ContainerStarted","Data":"1ddebb373cff93968a3b9d4c69bff2ae1b051547704836f0d76cd27adfcff520"}
May 25 13:24:30.414303 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:30.413996    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-696fc77849-9ntd8"
May 25 13:24:30.433811 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:30.433747    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/odh-model-controller-696fc77849-9ntd8" podStartSLOduration=1.921917309 podStartE2EDuration="4.43372955s" podCreationTimestamp="2026-05-25 13:24:26 +0000 UTC" firstStartedPulling="2026-05-25 13:24:27.154544252 +0000 UTC m=+1133.873507368" lastFinishedPulling="2026-05-25 13:24:29.666356487 +0000 UTC m=+1136.385319609" observedRunningTime="2026-05-25 13:24:30.43075959 +0000 UTC m=+1137.149722731" watchObservedRunningTime="2026-05-25 13:24:30.43372955 +0000 UTC m=+1137.152692682"
May 25 13:24:41.420953 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:41.420915    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-696fc77849-9ntd8"
May 25 13:24:42.271273 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:42.271235    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/s3-init-82g57"]
May 25 13:24:42.274800 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:42.274782    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-82g57"
May 25 13:24:42.283658 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:42.283629    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-82g57"]
May 25 13:24:42.361655 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:42.361620    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t22zj\" (UniqueName: \"kubernetes.io/projected/4304656e-7e95-4287-96d4-320989cd5888-kube-api-access-t22zj\") pod \"s3-init-82g57\" (UID: \"4304656e-7e95-4287-96d4-320989cd5888\") " pod="kserve/s3-init-82g57"
May 25 13:24:42.463072 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:42.463038    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t22zj\" (UniqueName: \"kubernetes.io/projected/4304656e-7e95-4287-96d4-320989cd5888-kube-api-access-t22zj\") pod \"s3-init-82g57\" (UID: \"4304656e-7e95-4287-96d4-320989cd5888\") " pod="kserve/s3-init-82g57"
May 25 13:24:42.471652 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:42.471620    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t22zj\" (UniqueName: \"kubernetes.io/projected/4304656e-7e95-4287-96d4-320989cd5888-kube-api-access-t22zj\") pod \"s3-init-82g57\" (UID: \"4304656e-7e95-4287-96d4-320989cd5888\") " pod="kserve/s3-init-82g57"
May 25 13:24:42.584570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:42.584484    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-82g57"
May 25 13:24:42.712020 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:42.711993    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-82g57"]
May 25 13:24:42.713707 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:24:42.713676    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4304656e_7e95_4287_96d4_320989cd5888.slice/crio-14f4624ecd49bf2f7d9adb736920a3a040cbf661a64fdf0c339fa17084a2853b WatchSource:0}: Error finding container 14f4624ecd49bf2f7d9adb736920a3a040cbf661a64fdf0c339fa17084a2853b: Status 404 returned error can't find the container with id 14f4624ecd49bf2f7d9adb736920a3a040cbf661a64fdf0c339fa17084a2853b
May 25 13:24:43.463253 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:43.463199    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-82g57" event={"ID":"4304656e-7e95-4287-96d4-320989cd5888","Type":"ContainerStarted","Data":"14f4624ecd49bf2f7d9adb736920a3a040cbf661a64fdf0c339fa17084a2853b"}
May 25 13:24:47.481414 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:47.481324    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-82g57" event={"ID":"4304656e-7e95-4287-96d4-320989cd5888","Type":"ContainerStarted","Data":"ab0da5bfac68ee81f733c1209b82470459eeb086f4ae118394afd43460bcab21"}
May 25 13:24:47.497834 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:47.497492    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/s3-init-82g57" podStartSLOduration=1.070151792 podStartE2EDuration="5.497475447s" podCreationTimestamp="2026-05-25 13:24:42 +0000 UTC" firstStartedPulling="2026-05-25 13:24:42.715591472 +0000 UTC m=+1149.434554588" lastFinishedPulling="2026-05-25 13:24:47.142915113 +0000 UTC m=+1153.861878243" observedRunningTime="2026-05-25 13:24:47.497186193 +0000 UTC m=+1154.216149333" watchObservedRunningTime="2026-05-25 13:24:47.497475447 +0000 UTC m=+1154.216438587"
May 25 13:24:50.496153 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:50.496118    2559 generic.go:358] "Generic (PLEG): container finished" podID="4304656e-7e95-4287-96d4-320989cd5888" containerID="ab0da5bfac68ee81f733c1209b82470459eeb086f4ae118394afd43460bcab21" exitCode=0
May 25 13:24:50.496506 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:50.496190    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-82g57" event={"ID":"4304656e-7e95-4287-96d4-320989cd5888","Type":"ContainerDied","Data":"ab0da5bfac68ee81f733c1209b82470459eeb086f4ae118394afd43460bcab21"}
May 25 13:24:51.633387 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:51.633360    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-82g57"
May 25 13:24:51.747562 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:51.747524    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t22zj\" (UniqueName: \"kubernetes.io/projected/4304656e-7e95-4287-96d4-320989cd5888-kube-api-access-t22zj\") pod \"4304656e-7e95-4287-96d4-320989cd5888\" (UID: \"4304656e-7e95-4287-96d4-320989cd5888\") "
May 25 13:24:51.749770 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:51.749738    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4304656e-7e95-4287-96d4-320989cd5888-kube-api-access-t22zj" (OuterVolumeSpecName: "kube-api-access-t22zj") pod "4304656e-7e95-4287-96d4-320989cd5888" (UID: "4304656e-7e95-4287-96d4-320989cd5888"). InnerVolumeSpecName "kube-api-access-t22zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:24:51.848417 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:51.848377    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t22zj\" (UniqueName: \"kubernetes.io/projected/4304656e-7e95-4287-96d4-320989cd5888-kube-api-access-t22zj\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:24:52.505133 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:52.505095    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-82g57" event={"ID":"4304656e-7e95-4287-96d4-320989cd5888","Type":"ContainerDied","Data":"14f4624ecd49bf2f7d9adb736920a3a040cbf661a64fdf0c339fa17084a2853b"}
May 25 13:24:52.505133 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:52.505131    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14f4624ecd49bf2f7d9adb736920a3a040cbf661a64fdf0c339fa17084a2853b"
May 25 13:24:52.505133 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:24:52.505134    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-82g57"
May 25 13:25:02.561625 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.561590    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"]
May 25 13:25:02.562112 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.562013    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4304656e-7e95-4287-96d4-320989cd5888" containerName="s3-init"
May 25 13:25:02.562112 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.562030    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4304656e-7e95-4287-96d4-320989cd5888" containerName="s3-init"
May 25 13:25:02.562112 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.562085    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="4304656e-7e95-4287-96d4-320989cd5888" containerName="s3-init"
May 25 13:25:02.566434 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.566411    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.569740 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.569358    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-gw-ca-root-cert\""
May 25 13:25:02.569740 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.569435    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
May 25 13:25:02.570014 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.569992    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
May 25 13:25:02.570219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.570202    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-1-openshift-default-dockercfg-mzk76\""
May 25 13:25:02.581828 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.581792    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"]
May 25 13:25:02.742001 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.741959    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-workload-certs\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.742001 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.742005    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istio-envoy\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.742226 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.742032    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istio-token\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.742226 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.742055    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.742226 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.742081    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istio-podinfo\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.742226 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.742121    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzhmp\" (UniqueName: \"kubernetes.io/projected/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-kube-api-access-hzhmp\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.742226 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.742170    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-workload-socket\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.742226 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.742224    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-credential-socket\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.742419 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.742272    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istio-data\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.843038 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.842939    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-workload-certs\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.843038 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.842991    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istio-envoy\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.843242 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.843120    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istio-token\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.843242 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.843166    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.843242 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.843203    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istio-podinfo\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.843391 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.843257    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hzhmp\" (UniqueName: \"kubernetes.io/projected/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-kube-api-access-hzhmp\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.843391 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.843290    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-workload-socket\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.843391 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.843352    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-credential-socket\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.843538 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.843410    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-workload-certs\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.843538 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.843410    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istio-data\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.843803 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.843781    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-workload-socket\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.843955 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.843929    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-credential-socket\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.844013 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.843941    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istio-data\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.844116 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.844086    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.846213 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.846189    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istio-envoy\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.846539 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.846515    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istio-podinfo\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.851681 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.851657    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-istio-token\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.851895 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.851880    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzhmp\" (UniqueName: \"kubernetes.io/projected/464b6cc1-f77b-4c8d-98e1-2c9fedfaa765-kube-api-access-hzhmp\") pod \"router-gateway-1-openshift-default-7b9cf87876-w7tqm\" (UID: \"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765\") " pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:02.878481 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:02.878448    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:03.033398 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:03.033360    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"]
May 25 13:25:03.033898 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:25:03.033866    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod464b6cc1_f77b_4c8d_98e1_2c9fedfaa765.slice/crio-b58a53990ae8703499559fdfaa5d1c2a24d33543c2fccfea5f12d5c443bdc832 WatchSource:0}: Error finding container b58a53990ae8703499559fdfaa5d1c2a24d33543c2fccfea5f12d5c443bdc832: Status 404 returned error can't find the container with id b58a53990ae8703499559fdfaa5d1c2a24d33543c2fccfea5f12d5c443bdc832
May 25 13:25:03.036288 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:03.036196    2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 13:25:03.036415 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:03.036285    2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 13:25:03.036415 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:03.036335    2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 13:25:03.549222 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:03.549183    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm" event={"ID":"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765","Type":"ContainerStarted","Data":"498fd2b72b3612ce74cdd63396688e60c55e4082c3616eb6a1bc5ff6cc96034d"}
May 25 13:25:03.549222 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:03.549225    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm" event={"ID":"464b6cc1-f77b-4c8d-98e1-2c9fedfaa765","Type":"ContainerStarted","Data":"b58a53990ae8703499559fdfaa5d1c2a24d33543c2fccfea5f12d5c443bdc832"}
May 25 13:25:03.569950 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:03.569895    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm" podStartSLOduration=1.5698789309999999 podStartE2EDuration="1.569878931s" podCreationTimestamp="2026-05-25 13:25:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:25:03.56896053 +0000 UTC m=+1170.287923704" watchObservedRunningTime="2026-05-25 13:25:03.569878931 +0000 UTC m=+1170.288842094"
May 25 13:25:03.879237 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:03.879155    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:04.884139 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:04.884108    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:05.557706 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:05.557674    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:05.558765 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:05.558744    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-1-openshift-default-7b9cf87876-w7tqm"
May 25 13:25:07.605683 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.605639    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"]
May 25 13:25:07.610566 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.610539    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.613408 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.613311    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-8blwv\""
May 25 13:25:07.613534 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.613312    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
May 25 13:25:07.615602 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.615577    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"]
May 25 13:25:07.677284 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.677241    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-dshm\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.677284 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.677283    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-tmp-dir\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.677492 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.677366    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.677492 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.677420    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gsms\" (UniqueName: \"kubernetes.io/projected/14282e22-71d2-481c-a176-c3d0ce2316a6-kube-api-access-6gsms\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.677492 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.677461    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-model-cache\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.677492 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.677485    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14282e22-71d2-481c-a176-c3d0ce2316a6-tls-certs\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.677621 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.677507    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-home\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.777971 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.777927    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6gsms\" (UniqueName: \"kubernetes.io/projected/14282e22-71d2-481c-a176-c3d0ce2316a6-kube-api-access-6gsms\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.778166 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.777980    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-model-cache\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.778166 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.778002    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14282e22-71d2-481c-a176-c3d0ce2316a6-tls-certs\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.778166 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.778033    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-home\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.778166 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.778062    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-dshm\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.778166 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.778077    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-tmp-dir\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.778166 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.778139    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.778503 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.778481    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-model-cache\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.778555 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.778536    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-home\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.778622 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.778565    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-kserve-provision-location\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.778673 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.778641    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-tmp-dir\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.780404 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.780382    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-dshm\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.780814 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.780793    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14282e22-71d2-481c-a176-c3d0ce2316a6-tls-certs\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.786356 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.786336    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gsms\" (UniqueName: \"kubernetes.io/projected/14282e22-71d2-481c-a176-c3d0ce2316a6-kube-api-access-6gsms\") pod \"scheduler-inline-config-test-kserve-754cf65554-2t868\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") " pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:07.926207 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:07.926108    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:25:08.067372 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:08.067336    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"]
May 25 13:25:08.067607 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:25:08.067577    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14282e22_71d2_481c_a176_c3d0ce2316a6.slice/crio-2a3390e5d7839946dfa3d87b8c90d44946f0a8b68b66d5afb64b4e1d418efe73 WatchSource:0}: Error finding container 2a3390e5d7839946dfa3d87b8c90d44946f0a8b68b66d5afb64b4e1d418efe73: Status 404 returned error can't find the container with id 2a3390e5d7839946dfa3d87b8c90d44946f0a8b68b66d5afb64b4e1d418efe73
May 25 13:25:08.570884 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:08.570823    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868" event={"ID":"14282e22-71d2-481c-a176-c3d0ce2316a6","Type":"ContainerStarted","Data":"2a3390e5d7839946dfa3d87b8c90d44946f0a8b68b66d5afb64b4e1d418efe73"}
May 25 13:25:12.588863 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:25:12.588826    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868" event={"ID":"14282e22-71d2-481c-a176-c3d0ce2316a6","Type":"ContainerStarted","Data":"6a9d60069035f29ad983e7974d0290f8cfac3134bd42300ccc2d9ce327c5d800"}
May 25 13:26:59.992323 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:26:59.992287    2559 generic.go:358] "Generic (PLEG): container finished" podID="14282e22-71d2-481c-a176-c3d0ce2316a6" containerID="6a9d60069035f29ad983e7974d0290f8cfac3134bd42300ccc2d9ce327c5d800" exitCode=0
May 25 13:26:59.992761 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:26:59.992358    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868" event={"ID":"14282e22-71d2-481c-a176-c3d0ce2316a6","Type":"ContainerDied","Data":"6a9d60069035f29ad983e7974d0290f8cfac3134bd42300ccc2d9ce327c5d800"}
May 25 13:27:02.006916 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:02.006881    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868" event={"ID":"14282e22-71d2-481c-a176-c3d0ce2316a6","Type":"ContainerStarted","Data":"59391010e28c7cd7088596edf6608f8bbca59a365fb810c146189d58f8c6855d"}
May 25 13:27:02.028057 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:02.028003    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868" podStartSLOduration=2.052449744 podStartE2EDuration="1m55.027985227s" podCreationTimestamp="2026-05-25 13:25:07 +0000 UTC" firstStartedPulling="2026-05-25 13:25:08.069458063 +0000 UTC m=+1174.788421180" lastFinishedPulling="2026-05-25 13:27:01.044993532 +0000 UTC m=+1287.763956663" observedRunningTime="2026-05-25 13:27:02.026769154 +0000 UTC m=+1288.745732292" watchObservedRunningTime="2026-05-25 13:27:02.027985227 +0000 UTC m=+1288.746948367"
May 25 13:27:07.926628 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:07.926590    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:27:07.927035 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:07.926644    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:27:07.939198 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:07.939174    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:27:08.041201 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:08.041173    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:27:09.108905 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:09.108872    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"]
May 25 13:27:09.164118 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:27:09.164079    2559 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs: secret "scheduler-inline-config-test-kserve-self-signed-certs" not found
May 25 13:27:09.164293 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:27:09.164255    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14282e22-71d2-481c-a176-c3d0ce2316a6-tls-certs podName:14282e22-71d2-481c-a176-c3d0ce2316a6 nodeName:}" failed. No retries permitted until 2026-05-25 13:27:09.664152907 +0000 UTC m=+1296.383116037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/14282e22-71d2-481c-a176-c3d0ce2316a6-tls-certs") pod "scheduler-inline-config-test-kserve-754cf65554-2t868" (UID: "14282e22-71d2-481c-a176-c3d0ce2316a6") : secret "scheduler-inline-config-test-kserve-self-signed-certs" not found
May 25 13:27:09.668431 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:27:09.668389    2559 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs: secret "scheduler-inline-config-test-kserve-self-signed-certs" not found
May 25 13:27:09.668633 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:27:09.668471    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14282e22-71d2-481c-a176-c3d0ce2316a6-tls-certs podName:14282e22-71d2-481c-a176-c3d0ce2316a6 nodeName:}" failed. No retries permitted until 2026-05-25 13:27:10.668452798 +0000 UTC m=+1297.387415919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/14282e22-71d2-481c-a176-c3d0ce2316a6-tls-certs") pod "scheduler-inline-config-test-kserve-754cf65554-2t868" (UID: "14282e22-71d2-481c-a176-c3d0ce2316a6") : secret "scheduler-inline-config-test-kserve-self-signed-certs" not found
May 25 13:27:10.041044 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.040996    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868" podUID="14282e22-71d2-481c-a176-c3d0ce2316a6" containerName="main" containerID="cri-o://59391010e28c7cd7088596edf6608f8bbca59a365fb810c146189d58f8c6855d" gracePeriod=30
May 25 13:27:10.330465 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.330443    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:27:10.473696 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.473662    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-home\") pod \"14282e22-71d2-481c-a176-c3d0ce2316a6\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") "
May 25 13:27:10.473917 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.473710    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-kserve-provision-location\") pod \"14282e22-71d2-481c-a176-c3d0ce2316a6\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") "
May 25 13:27:10.473917 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.473776    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-tmp-dir\") pod \"14282e22-71d2-481c-a176-c3d0ce2316a6\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") "
May 25 13:27:10.473917 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.473831    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14282e22-71d2-481c-a176-c3d0ce2316a6-tls-certs\") pod \"14282e22-71d2-481c-a176-c3d0ce2316a6\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") "
May 25 13:27:10.473917 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.473855    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-model-cache\") pod \"14282e22-71d2-481c-a176-c3d0ce2316a6\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") "
May 25 13:27:10.473917 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.473885    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-dshm\") pod \"14282e22-71d2-481c-a176-c3d0ce2316a6\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") "
May 25 13:27:10.474177 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.473925    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gsms\" (UniqueName: \"kubernetes.io/projected/14282e22-71d2-481c-a176-c3d0ce2316a6-kube-api-access-6gsms\") pod \"14282e22-71d2-481c-a176-c3d0ce2316a6\" (UID: \"14282e22-71d2-481c-a176-c3d0ce2316a6\") "
May 25 13:27:10.474177 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.474010    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-home" (OuterVolumeSpecName: "home") pod "14282e22-71d2-481c-a176-c3d0ce2316a6" (UID: "14282e22-71d2-481c-a176-c3d0ce2316a6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:27:10.474177 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.474134    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "14282e22-71d2-481c-a176-c3d0ce2316a6" (UID: "14282e22-71d2-481c-a176-c3d0ce2316a6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:27:10.474177 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.474148    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-model-cache" (OuterVolumeSpecName: "model-cache") pod "14282e22-71d2-481c-a176-c3d0ce2316a6" (UID: "14282e22-71d2-481c-a176-c3d0ce2316a6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:27:10.474366 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.474226    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-tmp-dir\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:27:10.474366 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.474246    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-model-cache\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:27:10.474366 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.474256    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-home\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:27:10.476190 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.476152    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-dshm" (OuterVolumeSpecName: "dshm") pod "14282e22-71d2-481c-a176-c3d0ce2316a6" (UID: "14282e22-71d2-481c-a176-c3d0ce2316a6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:27:10.476302 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.476206    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14282e22-71d2-481c-a176-c3d0ce2316a6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "14282e22-71d2-481c-a176-c3d0ce2316a6" (UID: "14282e22-71d2-481c-a176-c3d0ce2316a6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:27:10.476302 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.476226    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14282e22-71d2-481c-a176-c3d0ce2316a6-kube-api-access-6gsms" (OuterVolumeSpecName: "kube-api-access-6gsms") pod "14282e22-71d2-481c-a176-c3d0ce2316a6" (UID: "14282e22-71d2-481c-a176-c3d0ce2316a6"). InnerVolumeSpecName "kube-api-access-6gsms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:27:10.575183 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.575100    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14282e22-71d2-481c-a176-c3d0ce2316a6-tls-certs\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:27:10.575183 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.575126    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-dshm\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:27:10.575183 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:10.575135    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gsms\" (UniqueName: \"kubernetes.io/projected/14282e22-71d2-481c-a176-c3d0ce2316a6-kube-api-access-6gsms\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:27:11.046085 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.046048    2559 generic.go:358] "Generic (PLEG): container finished" podID="14282e22-71d2-481c-a176-c3d0ce2316a6" containerID="59391010e28c7cd7088596edf6608f8bbca59a365fb810c146189d58f8c6855d" exitCode=0
May 25 13:27:11.046261 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.046084    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868" event={"ID":"14282e22-71d2-481c-a176-c3d0ce2316a6","Type":"ContainerDied","Data":"59391010e28c7cd7088596edf6608f8bbca59a365fb810c146189d58f8c6855d"}
May 25 13:27:11.046261 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.046111    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"
May 25 13:27:11.046261 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.046124    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868" event={"ID":"14282e22-71d2-481c-a176-c3d0ce2316a6","Type":"ContainerDied","Data":"2a3390e5d7839946dfa3d87b8c90d44946f0a8b68b66d5afb64b4e1d418efe73"}
May 25 13:27:11.046261 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.046141    2559 scope.go:117] "RemoveContainer" containerID="59391010e28c7cd7088596edf6608f8bbca59a365fb810c146189d58f8c6855d"
May 25 13:27:11.054666 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.054643    2559 scope.go:117] "RemoveContainer" containerID="6a9d60069035f29ad983e7974d0290f8cfac3134bd42300ccc2d9ce327c5d800"
May 25 13:27:11.115477 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.115455    2559 scope.go:117] "RemoveContainer" containerID="59391010e28c7cd7088596edf6608f8bbca59a365fb810c146189d58f8c6855d"
May 25 13:27:11.115781 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:27:11.115759    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59391010e28c7cd7088596edf6608f8bbca59a365fb810c146189d58f8c6855d\": container with ID starting with 59391010e28c7cd7088596edf6608f8bbca59a365fb810c146189d58f8c6855d not found: ID does not exist" containerID="59391010e28c7cd7088596edf6608f8bbca59a365fb810c146189d58f8c6855d"
May 25 13:27:11.115831 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.115790    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59391010e28c7cd7088596edf6608f8bbca59a365fb810c146189d58f8c6855d"} err="failed to get container status \"59391010e28c7cd7088596edf6608f8bbca59a365fb810c146189d58f8c6855d\": rpc error: code = NotFound desc = could not find container \"59391010e28c7cd7088596edf6608f8bbca59a365fb810c146189d58f8c6855d\": container with ID starting with 59391010e28c7cd7088596edf6608f8bbca59a365fb810c146189d58f8c6855d not found: ID does not exist"
May 25 13:27:11.115831 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.115809    2559 scope.go:117] "RemoveContainer" containerID="6a9d60069035f29ad983e7974d0290f8cfac3134bd42300ccc2d9ce327c5d800"
May 25 13:27:11.116079 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:27:11.116062    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a9d60069035f29ad983e7974d0290f8cfac3134bd42300ccc2d9ce327c5d800\": container with ID starting with 6a9d60069035f29ad983e7974d0290f8cfac3134bd42300ccc2d9ce327c5d800 not found: ID does not exist" containerID="6a9d60069035f29ad983e7974d0290f8cfac3134bd42300ccc2d9ce327c5d800"
May 25 13:27:11.116115 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.116090    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a9d60069035f29ad983e7974d0290f8cfac3134bd42300ccc2d9ce327c5d800"} err="failed to get container status \"6a9d60069035f29ad983e7974d0290f8cfac3134bd42300ccc2d9ce327c5d800\": rpc error: code = NotFound desc = could not find container \"6a9d60069035f29ad983e7974d0290f8cfac3134bd42300ccc2d9ce327c5d800\": container with ID starting with 6a9d60069035f29ad983e7974d0290f8cfac3134bd42300ccc2d9ce327c5d800 not found: ID does not exist"
May 25 13:27:11.513596 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.513539    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "14282e22-71d2-481c-a176-c3d0ce2316a6" (UID: "14282e22-71d2-481c-a176-c3d0ce2316a6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:27:11.583768 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.583700    2559 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/14282e22-71d2-481c-a176-c3d0ce2316a6-kserve-provision-location\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:27:11.669352 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.669315    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"]
May 25 13:27:11.674945 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.674920    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-inline-config-test-kserve-754cf65554-2t868"]
May 25 13:27:11.869270 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:11.869178    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14282e22-71d2-481c-a176-c3d0ce2316a6" path="/var/lib/kubelet/pods/14282e22-71d2-481c-a176-c3d0ce2316a6/volumes"
May 25 13:27:17.712212 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.712175    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"]
May 25 13:27:17.712590 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.712519    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14282e22-71d2-481c-a176-c3d0ce2316a6" containerName="main"
May 25 13:27:17.712590 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.712530    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="14282e22-71d2-481c-a176-c3d0ce2316a6" containerName="main"
May 25 13:27:17.712590 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.712548    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14282e22-71d2-481c-a176-c3d0ce2316a6" containerName="storage-initializer"
May 25 13:27:17.712590 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.712553    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="14282e22-71d2-481c-a176-c3d0ce2316a6" containerName="storage-initializer"
May 25 13:27:17.712792 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.712606    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="14282e22-71d2-481c-a176-c3d0ce2316a6" containerName="main"
May 25 13:27:17.715801 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.715778    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.718695 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.718676    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
May 25 13:27:17.718825 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.718696    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-8blwv\""
May 25 13:27:17.727147 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.727125    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"]
May 25 13:27:17.837953 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.837912    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h77zl\" (UniqueName: \"kubernetes.io/projected/4b77229a-101d-42ca-8633-16287dfb0e2b-kube-api-access-h77zl\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.838134 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.837983    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.838134 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.838017    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-home\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.838134 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.838058    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-model-cache\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.838307 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.838154    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-dshm\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.838307 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.838285    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b77229a-101d-42ca-8633-16287dfb0e2b-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.838408 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.838376    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.939218 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.939176    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b77229a-101d-42ca-8633-16287dfb0e2b-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.939409 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.939248    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.939409 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.939281    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h77zl\" (UniqueName: \"kubernetes.io/projected/4b77229a-101d-42ca-8633-16287dfb0e2b-kube-api-access-h77zl\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.939409 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.939315    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.939409 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.939341    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-home\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.939409 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.939378    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-model-cache\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.939667 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.939415    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-dshm\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.939814 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.939785    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-kserve-provision-location\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.939973 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.939934    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-home\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.940096 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.939971    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-model-cache\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.940096 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.940057    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.941825 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.941805    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-dshm\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.941998 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.941977    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b77229a-101d-42ca-8633-16287dfb0e2b-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:17.950073 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:17.950054    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h77zl\" (UniqueName: \"kubernetes.io/projected/4b77229a-101d-42ca-8633-16287dfb0e2b-kube-api-access-h77zl\") pod \"scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") " pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:18.026906 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:18.026872    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:18.162262 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:18.162233    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"]
May 25 13:27:18.163672 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:27:18.163643    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b77229a_101d_42ca_8633_16287dfb0e2b.slice/crio-3912211ae64a3bf332c9c38af51103bb1cb29cf84b25e5c88cb3a642c836c19a WatchSource:0}: Error finding container 3912211ae64a3bf332c9c38af51103bb1cb29cf84b25e5c88cb3a642c836c19a: Status 404 returned error can't find the container with id 3912211ae64a3bf332c9c38af51103bb1cb29cf84b25e5c88cb3a642c836c19a
May 25 13:27:19.076101 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:19.076057    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69" event={"ID":"4b77229a-101d-42ca-8633-16287dfb0e2b","Type":"ContainerStarted","Data":"aec674d2aeef01bf4917cf688a538de99475287c05479e71d9ca398dc0617fc6"}
May 25 13:27:19.076553 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:19.076111    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69" event={"ID":"4b77229a-101d-42ca-8633-16287dfb0e2b","Type":"ContainerStarted","Data":"3912211ae64a3bf332c9c38af51103bb1cb29cf84b25e5c88cb3a642c836c19a"}
May 25 13:27:24.100204 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:24.100168    2559 generic.go:358] "Generic (PLEG): container finished" podID="4b77229a-101d-42ca-8633-16287dfb0e2b" containerID="aec674d2aeef01bf4917cf688a538de99475287c05479e71d9ca398dc0617fc6" exitCode=0
May 25 13:27:24.100609 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:24.100243    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69" event={"ID":"4b77229a-101d-42ca-8633-16287dfb0e2b","Type":"ContainerDied","Data":"aec674d2aeef01bf4917cf688a538de99475287c05479e71d9ca398dc0617fc6"}
May 25 13:27:25.105492 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:25.105458    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69" event={"ID":"4b77229a-101d-42ca-8633-16287dfb0e2b","Type":"ContainerStarted","Data":"04affd1e3f71ce5e1ad745862ffb3761143b940620c7256dc3ae97df0730188a"}
May 25 13:27:25.124621 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:25.124575    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69" podStartSLOduration=8.124562961 podStartE2EDuration="8.124562961s" podCreationTimestamp="2026-05-25 13:27:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:27:25.121963238 +0000 UTC m=+1311.840926377" watchObservedRunningTime="2026-05-25 13:27:25.124562961 +0000 UTC m=+1311.843526100"
May 25 13:27:28.028091 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:28.028049    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:28.030454 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:28.028122    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:28.040611 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:28.040591    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:27:28.127592 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:27:28.127568    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:28:01.764438 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:01.764354    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"]
May 25 13:28:01.764942 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:01.764662    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69" podUID="4b77229a-101d-42ca-8633-16287dfb0e2b" containerName="main" containerID="cri-o://04affd1e3f71ce5e1ad745862ffb3761143b940620c7256dc3ae97df0730188a" gracePeriod=30
May 25 13:28:02.015938 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.015870    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:28:02.105495 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.105464    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-tmp-dir\") pod \"4b77229a-101d-42ca-8633-16287dfb0e2b\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") "
May 25 13:28:02.105495 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.105498    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-home\") pod \"4b77229a-101d-42ca-8633-16287dfb0e2b\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") "
May 25 13:28:02.105936 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.105518    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-kserve-provision-location\") pod \"4b77229a-101d-42ca-8633-16287dfb0e2b\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") "
May 25 13:28:02.105936 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.105541    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h77zl\" (UniqueName: \"kubernetes.io/projected/4b77229a-101d-42ca-8633-16287dfb0e2b-kube-api-access-h77zl\") pod \"4b77229a-101d-42ca-8633-16287dfb0e2b\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") "
May 25 13:28:02.105936 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.105555    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-dshm\") pod \"4b77229a-101d-42ca-8633-16287dfb0e2b\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") "
May 25 13:28:02.105936 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.105585    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b77229a-101d-42ca-8633-16287dfb0e2b-tls-certs\") pod \"4b77229a-101d-42ca-8633-16287dfb0e2b\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") "
May 25 13:28:02.105936 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.105641    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-model-cache\") pod \"4b77229a-101d-42ca-8633-16287dfb0e2b\" (UID: \"4b77229a-101d-42ca-8633-16287dfb0e2b\") "
May 25 13:28:02.105936 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.105833    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-home" (OuterVolumeSpecName: "home") pod "4b77229a-101d-42ca-8633-16287dfb0e2b" (UID: "4b77229a-101d-42ca-8633-16287dfb0e2b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:28:02.105936 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.105840    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4b77229a-101d-42ca-8633-16287dfb0e2b" (UID: "4b77229a-101d-42ca-8633-16287dfb0e2b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:28:02.106306 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.105944    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-tmp-dir\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:28:02.106306 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.105966    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-home\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:28:02.106306 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.106028    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-model-cache" (OuterVolumeSpecName: "model-cache") pod "4b77229a-101d-42ca-8633-16287dfb0e2b" (UID: "4b77229a-101d-42ca-8633-16287dfb0e2b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:28:02.107766 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.107738    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-dshm" (OuterVolumeSpecName: "dshm") pod "4b77229a-101d-42ca-8633-16287dfb0e2b" (UID: "4b77229a-101d-42ca-8633-16287dfb0e2b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:28:02.107766 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.107753    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b77229a-101d-42ca-8633-16287dfb0e2b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4b77229a-101d-42ca-8633-16287dfb0e2b" (UID: "4b77229a-101d-42ca-8633-16287dfb0e2b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:28:02.107917 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.107848    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b77229a-101d-42ca-8633-16287dfb0e2b-kube-api-access-h77zl" (OuterVolumeSpecName: "kube-api-access-h77zl") pod "4b77229a-101d-42ca-8633-16287dfb0e2b" (UID: "4b77229a-101d-42ca-8633-16287dfb0e2b"). InnerVolumeSpecName "kube-api-access-h77zl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:28:02.162862 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.162814    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4b77229a-101d-42ca-8633-16287dfb0e2b" (UID: "4b77229a-101d-42ca-8633-16287dfb0e2b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:28:02.206911 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.206875    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h77zl\" (UniqueName: \"kubernetes.io/projected/4b77229a-101d-42ca-8633-16287dfb0e2b-kube-api-access-h77zl\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:28:02.206911 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.206903    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-dshm\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:28:02.206911 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.206914    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b77229a-101d-42ca-8633-16287dfb0e2b-tls-certs\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:28:02.207199 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.206923    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-model-cache\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:28:02.207199 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.206931    2559 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4b77229a-101d-42ca-8633-16287dfb0e2b-kserve-provision-location\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:28:02.248504 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.248468    2559 generic.go:358] "Generic (PLEG): container finished" podID="4b77229a-101d-42ca-8633-16287dfb0e2b" containerID="04affd1e3f71ce5e1ad745862ffb3761143b940620c7256dc3ae97df0730188a" exitCode=0
May 25 13:28:02.248696 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.248560    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"
May 25 13:28:02.248696 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.248558    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69" event={"ID":"4b77229a-101d-42ca-8633-16287dfb0e2b","Type":"ContainerDied","Data":"04affd1e3f71ce5e1ad745862ffb3761143b940620c7256dc3ae97df0730188a"}
May 25 13:28:02.248696 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.248605    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69" event={"ID":"4b77229a-101d-42ca-8633-16287dfb0e2b","Type":"ContainerDied","Data":"3912211ae64a3bf332c9c38af51103bb1cb29cf84b25e5c88cb3a642c836c19a"}
May 25 13:28:02.248696 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.248625    2559 scope.go:117] "RemoveContainer" containerID="04affd1e3f71ce5e1ad745862ffb3761143b940620c7256dc3ae97df0730188a"
May 25 13:28:02.257982 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.257965    2559 scope.go:117] "RemoveContainer" containerID="aec674d2aeef01bf4917cf688a538de99475287c05479e71d9ca398dc0617fc6"
May 25 13:28:02.273136 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.273064    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"]
May 25 13:28:02.274073 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.274057    2559 scope.go:117] "RemoveContainer" containerID="04affd1e3f71ce5e1ad745862ffb3761143b940620c7256dc3ae97df0730188a"
May 25 13:28:02.274418 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:28:02.274390    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04affd1e3f71ce5e1ad745862ffb3761143b940620c7256dc3ae97df0730188a\": container with ID starting with 04affd1e3f71ce5e1ad745862ffb3761143b940620c7256dc3ae97df0730188a not found: ID does not exist" containerID="04affd1e3f71ce5e1ad745862ffb3761143b940620c7256dc3ae97df0730188a"
May 25 13:28:02.274537 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.274424    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04affd1e3f71ce5e1ad745862ffb3761143b940620c7256dc3ae97df0730188a"} err="failed to get container status \"04affd1e3f71ce5e1ad745862ffb3761143b940620c7256dc3ae97df0730188a\": rpc error: code = NotFound desc = could not find container \"04affd1e3f71ce5e1ad745862ffb3761143b940620c7256dc3ae97df0730188a\": container with ID starting with 04affd1e3f71ce5e1ad745862ffb3761143b940620c7256dc3ae97df0730188a not found: ID does not exist"
May 25 13:28:02.274537 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.274449    2559 scope.go:117] "RemoveContainer" containerID="aec674d2aeef01bf4917cf688a538de99475287c05479e71d9ca398dc0617fc6"
May 25 13:28:02.274944 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:28:02.274917    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aec674d2aeef01bf4917cf688a538de99475287c05479e71d9ca398dc0617fc6\": container with ID starting with aec674d2aeef01bf4917cf688a538de99475287c05479e71d9ca398dc0617fc6 not found: ID does not exist" containerID="aec674d2aeef01bf4917cf688a538de99475287c05479e71d9ca398dc0617fc6"
May 25 13:28:02.275094 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.274982    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aec674d2aeef01bf4917cf688a538de99475287c05479e71d9ca398dc0617fc6"} err="failed to get container status \"aec674d2aeef01bf4917cf688a538de99475287c05479e71d9ca398dc0617fc6\": rpc error: code = NotFound desc = could not find container \"aec674d2aeef01bf4917cf688a538de99475287c05479e71d9ca398dc0617fc6\": container with ID starting with aec674d2aeef01bf4917cf688a538de99475287c05479e71d9ca398dc0617fc6 not found: ID does not exist"
May 25 13:28:02.278426 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:02.278402    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-7d4dc488b6-mrs69"]
May 25 13:28:03.869948 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:03.869911    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b77229a-101d-42ca-8633-16287dfb0e2b" path="/var/lib/kubelet/pods/4b77229a-101d-42ca-8633-16287dfb0e2b/volumes"
May 25 13:28:08.632217 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.632179    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"]
May 25 13:28:08.632709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.632536    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b77229a-101d-42ca-8633-16287dfb0e2b" containerName="main"
May 25 13:28:08.632709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.632551    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b77229a-101d-42ca-8633-16287dfb0e2b" containerName="main"
May 25 13:28:08.632709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.632568    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b77229a-101d-42ca-8633-16287dfb0e2b" containerName="storage-initializer"
May 25 13:28:08.632709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.632576    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b77229a-101d-42ca-8633-16287dfb0e2b" containerName="storage-initializer"
May 25 13:28:08.632709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.632644    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="4b77229a-101d-42ca-8633-16287dfb0e2b" containerName="main"
May 25 13:28:08.636085 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.636065    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.638648 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.638628    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-8blwv\""
May 25 13:28:08.639508 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.639491    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
May 25 13:28:08.648384 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.648362    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"]
May 25 13:28:08.764517 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.764483    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-model-cache\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.764691 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.764537    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.764691 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.764601    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.764691 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.764636    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.764691 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.764658    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-dshm\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.764870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.764701    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h76x6\" (UniqueName: \"kubernetes.io/projected/ecdfb810-4349-4404-a1ad-1cf0528f45aa-kube-api-access-h76x6\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.764870 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.764776    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-home\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.865171 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.865137    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h76x6\" (UniqueName: \"kubernetes.io/projected/ecdfb810-4349-4404-a1ad-1cf0528f45aa-kube-api-access-h76x6\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.865171 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.865178    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-home\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.865415 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.865208    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-model-cache\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.865415 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.865239    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.865415 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.865260    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.865415 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.865276    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.865611 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.865446    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-dshm\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.865656 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.865618    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.865709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.865679    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-home\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.865801 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.865747    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-model-cache\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.865860 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.865795    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.867638 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.867616    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-dshm\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.867844 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.867826    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.874511 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.874486    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h76x6\" (UniqueName: \"kubernetes.io/projected/ecdfb810-4349-4404-a1ad-1cf0528f45aa-kube-api-access-h76x6\") pod \"scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.946664 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.946573    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:28:08.988445 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.987475    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"]
May 25 13:28:08.991167 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:08.991142    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.009355 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.009201    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-kv9g7\""
May 25 13:28:09.058700 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.058663    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"]
May 25 13:28:09.067504 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.067465    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.067684 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.067601    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.067775 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.067695    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.067775 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.067743    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.067893 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.067777    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.067893 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.067820    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frvff\" (UniqueName: \"kubernetes.io/projected/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-kube-api-access-frvff\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.147052 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.147020    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"]
May 25 13:28:09.147469 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:28:09.147447    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecdfb810_4349_4404_a1ad_1cf0528f45aa.slice/crio-98caafab7e9be2f3e6957e782cf5807479dd7082d00e731e5231823a00db2992 WatchSource:0}: Error finding container 98caafab7e9be2f3e6957e782cf5807479dd7082d00e731e5231823a00db2992: Status 404 returned error can't find the container with id 98caafab7e9be2f3e6957e782cf5807479dd7082d00e731e5231823a00db2992
May 25 13:28:09.168533 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.168505    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.168654 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.168542    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.168654 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.168564    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.168654 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.168580    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-frvff\" (UniqueName: \"kubernetes.io/projected/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-kube-api-access-frvff\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.168654 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.168616    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.168874 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.168798    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.168932 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.168909    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-cache\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.168932 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.168919    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-uds\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.169025 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.168977    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-kserve-provision-location\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.169025 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.168996    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-tmp\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.171093 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.171069    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.185189 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.185160    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frvff\" (UniqueName: \"kubernetes.io/projected/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-kube-api-access-frvff\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") " pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.280396 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.280337    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9" event={"ID":"ecdfb810-4349-4404-a1ad-1cf0528f45aa","Type":"ContainerStarted","Data":"30a46347929df96b41db4de95af3559283b497ccc5a509eebc2d2f1cb33cd008"}
May 25 13:28:09.280396 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.280390    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9" event={"ID":"ecdfb810-4349-4404-a1ad-1cf0528f45aa","Type":"ContainerStarted","Data":"98caafab7e9be2f3e6957e782cf5807479dd7082d00e731e5231823a00db2992"}
May 25 13:28:09.303666 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.303633    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:09.455320 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:09.455294    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"]
May 25 13:28:09.457494 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:28:09.457464    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf54e3aac_fb8a_4c0a_8fcf_b3f7c6a98245.slice/crio-9238ce2f0f977f78928b2d07383f3127c5693c9f4164b27827965483331d739f WatchSource:0}: Error finding container 9238ce2f0f977f78928b2d07383f3127c5693c9f4164b27827965483331d739f: Status 404 returned error can't find the container with id 9238ce2f0f977f78928b2d07383f3127c5693c9f4164b27827965483331d739f
May 25 13:28:10.285809 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:10.285744    2559 generic.go:358] "Generic (PLEG): container finished" podID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerID="717932b159e0b9503c76be37484d81015b9c1e4102755d1017bf864773996f97" exitCode=0
May 25 13:28:10.285809 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:10.285840    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" event={"ID":"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245","Type":"ContainerDied","Data":"717932b159e0b9503c76be37484d81015b9c1e4102755d1017bf864773996f97"}
May 25 13:28:10.286494 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:10.285879    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" event={"ID":"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245","Type":"ContainerStarted","Data":"9238ce2f0f977f78928b2d07383f3127c5693c9f4164b27827965483331d739f"}
May 25 13:28:12.297533 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:12.297491    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" event={"ID":"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245","Type":"ContainerStarted","Data":"0c841971456567dd576d9a4058036d134c2b20bcb8e059ddec30a453d26a0695"}
May 25 13:28:42.440143 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:42.440099    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" event={"ID":"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245","Type":"ContainerStarted","Data":"5adf6718afb477abf3ae73d3773ad5c99a29d8944a326e66811ea87e0e9ae06c"}
May 25 13:28:42.440584 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:42.440321    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:42.443096 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:42.443067    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:28:42.463435 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:42.463378    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podStartSLOduration=3.017526053 podStartE2EDuration="34.463362728s" podCreationTimestamp="2026-05-25 13:28:08 +0000 UTC" firstStartedPulling="2026-05-25 13:28:10.287409447 +0000 UTC m=+1357.006372564" lastFinishedPulling="2026-05-25 13:28:41.733246119 +0000 UTC m=+1388.452209239" observedRunningTime="2026-05-25 13:28:42.460511565 +0000 UTC m=+1389.179474705" watchObservedRunningTime="2026-05-25 13:28:42.463362728 +0000 UTC m=+1389.182325871"
May 25 13:28:43.445616 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:43.445582    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:28:49.304385 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:49.304347    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:49.304385 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:49.304392    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:49.304922 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:49.304805    2559 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="tokenizer" probeResult="failure" output="Get \"http://10.133.0.50:8082/healthz\": dial tcp 10.133.0.50:8082: connect: connection refused"
May 25 13:28:49.305774 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:49.305747    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:28:59.305480 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:59.305444    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:59.305965 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:59.305860    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:28:59.306791 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:59.306767    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:28:59.307056 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:59.307028    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:28:59.506678 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:28:59.506644    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:29:09.507405 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:29:09.507365    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:29:19.507035 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:29:19.506948    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:29:29.506709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:29:29.506669    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:29:39.507217 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:29:39.507177    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:29:49.507570 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:29:49.507530    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:29:59.507449 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:29:59.507410    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:30:09.507051 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:09.507013    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:30:10.866368 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:10.866334    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:30:20.866945 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:20.866901    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:30:30.866434 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:30.866394    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:30:32.859009 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:32.858968    2559 generic.go:358] "Generic (PLEG): container finished" podID="ecdfb810-4349-4404-a1ad-1cf0528f45aa" containerID="30a46347929df96b41db4de95af3559283b497ccc5a509eebc2d2f1cb33cd008" exitCode=0
May 25 13:30:32.859385 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:32.859040    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9" event={"ID":"ecdfb810-4349-4404-a1ad-1cf0528f45aa","Type":"ContainerDied","Data":"30a46347929df96b41db4de95af3559283b497ccc5a509eebc2d2f1cb33cd008"}
May 25 13:30:32.860156 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:32.860139    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 13:30:33.863944 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:33.863902    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9" event={"ID":"ecdfb810-4349-4404-a1ad-1cf0528f45aa","Type":"ContainerStarted","Data":"3cbca2e1b874597c1667c8deceb89a4721144bd68e2965acf269507af1e210cd"}
May 25 13:30:33.888072 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:33.888018    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9" podStartSLOduration=145.888000711 podStartE2EDuration="2m25.888000711s" podCreationTimestamp="2026-05-25 13:28:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:30:33.885633633 +0000 UTC m=+1500.604596772" watchObservedRunningTime="2026-05-25 13:30:33.888000711 +0000 UTC m=+1500.606963853"
May 25 13:30:38.947633 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:38.947595    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:30:38.947633 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:38.947630    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:30:38.959947 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:38.959924    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:30:39.896620 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:39.896594    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:30:40.866385 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:40.866349    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:30:41.006350 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:41.006313    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"]
May 25 13:30:41.006857 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:41.006697    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="tokenizer" containerID="cri-o://5adf6718afb477abf3ae73d3773ad5c99a29d8944a326e66811ea87e0e9ae06c" gracePeriod=30
May 25 13:30:41.006857 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:41.006672    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" containerID="cri-o://0c841971456567dd576d9a4058036d134c2b20bcb8e059ddec30a453d26a0695" gracePeriod=30
May 25 13:30:41.008162 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:41.008133    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
May 25 13:30:41.015957 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:41.015932    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"]
May 25 13:30:41.040668 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:30:41.040637    2559 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs: secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
May 25 13:30:41.040891 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:30:41.040741    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tls-certs podName:ecdfb810-4349-4404-a1ad-1cf0528f45aa nodeName:}" failed. No retries permitted until 2026-05-25 13:30:41.540701444 +0000 UTC m=+1508.259664560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tls-certs") pod "scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9" (UID: "ecdfb810-4349-4404-a1ad-1cf0528f45aa") : secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
May 25 13:30:41.545383 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:30:41.545349    2559 secret.go:189] Couldn't get secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs: secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
May 25 13:30:41.545572 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:30:41.545426    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tls-certs podName:ecdfb810-4349-4404-a1ad-1cf0528f45aa nodeName:}" failed. No retries permitted until 2026-05-25 13:30:42.545408508 +0000 UTC m=+1509.264371625 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tls-certs") pod "scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9" (UID: "ecdfb810-4349-4404-a1ad-1cf0528f45aa") : secret "scheduler-ha-replicas-test-kserve-self-signed-certs" not found
May 25 13:30:41.896379 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:41.896295    2559 generic.go:358] "Generic (PLEG): container finished" podID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerID="0c841971456567dd576d9a4058036d134c2b20bcb8e059ddec30a453d26a0695" exitCode=0
May 25 13:30:41.896840 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:41.896373    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" event={"ID":"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245","Type":"ContainerDied","Data":"0c841971456567dd576d9a4058036d134c2b20bcb8e059ddec30a453d26a0695"}
May 25 13:30:41.896840 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:41.896770    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9" podUID="ecdfb810-4349-4404-a1ad-1cf0528f45aa" containerName="main" containerID="cri-o://3cbca2e1b874597c1667c8deceb89a4721144bd68e2965acf269507af1e210cd" gracePeriod=30
May 25 13:30:42.168010 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.167985    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:30:42.252454 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.252422    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-kserve-provision-location\") pod \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") "
May 25 13:30:42.252634 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.252468    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-model-cache\") pod \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") "
May 25 13:30:42.252634 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.252514    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-home\") pod \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") "
May 25 13:30:42.252634 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.252549    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tmp-dir\") pod \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") "
May 25 13:30:42.252634 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.252580    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tls-certs\") pod \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") "
May 25 13:30:42.252634 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.252620    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h76x6\" (UniqueName: \"kubernetes.io/projected/ecdfb810-4349-4404-a1ad-1cf0528f45aa-kube-api-access-h76x6\") pod \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") "
May 25 13:30:42.252929 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.252789    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-model-cache" (OuterVolumeSpecName: "model-cache") pod "ecdfb810-4349-4404-a1ad-1cf0528f45aa" (UID: "ecdfb810-4349-4404-a1ad-1cf0528f45aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:30:42.252929 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.252872    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ecdfb810-4349-4404-a1ad-1cf0528f45aa" (UID: "ecdfb810-4349-4404-a1ad-1cf0528f45aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:30:42.253044 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.252965    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-dshm\") pod \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\" (UID: \"ecdfb810-4349-4404-a1ad-1cf0528f45aa\") "
May 25 13:30:42.253292 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.253274    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-model-cache\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:30:42.253362 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.253300    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tmp-dir\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:30:42.254052 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.254022    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-home" (OuterVolumeSpecName: "home") pod "ecdfb810-4349-4404-a1ad-1cf0528f45aa" (UID: "ecdfb810-4349-4404-a1ad-1cf0528f45aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:30:42.254901 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.254877    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ecdfb810-4349-4404-a1ad-1cf0528f45aa" (UID: "ecdfb810-4349-4404-a1ad-1cf0528f45aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:30:42.255041 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.255027    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-dshm" (OuterVolumeSpecName: "dshm") pod "ecdfb810-4349-4404-a1ad-1cf0528f45aa" (UID: "ecdfb810-4349-4404-a1ad-1cf0528f45aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:30:42.255573 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.255556    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecdfb810-4349-4404-a1ad-1cf0528f45aa-kube-api-access-h76x6" (OuterVolumeSpecName: "kube-api-access-h76x6") pod "ecdfb810-4349-4404-a1ad-1cf0528f45aa" (UID: "ecdfb810-4349-4404-a1ad-1cf0528f45aa"). InnerVolumeSpecName "kube-api-access-h76x6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:30:42.275497 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.275475    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:30:42.314741 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.314699    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ecdfb810-4349-4404-a1ad-1cf0528f45aa" (UID: "ecdfb810-4349-4404-a1ad-1cf0528f45aa"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:30:42.354416 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354382    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-kserve-provision-location\") pod \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") "
May 25 13:30:42.354539 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354424    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-cache\") pod \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") "
May 25 13:30:42.354539 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354447    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tls-certs\") pod \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") "
May 25 13:30:42.354539 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354475    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frvff\" (UniqueName: \"kubernetes.io/projected/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-kube-api-access-frvff\") pod \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") "
May 25 13:30:42.354539 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354523    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-uds\") pod \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") "
May 25 13:30:42.354799 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354552    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-tmp\") pod \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\" (UID: \"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245\") "
May 25 13:30:42.354799 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354674    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" (UID: "f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:30:42.354902 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354863    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" (UID: "f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:30:42.354902 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354885    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-home\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:30:42.354989 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354907    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecdfb810-4349-4404-a1ad-1cf0528f45aa-tls-certs\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:30:42.354989 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354918    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h76x6\" (UniqueName: \"kubernetes.io/projected/ecdfb810-4349-4404-a1ad-1cf0528f45aa-kube-api-access-h76x6\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:30:42.354989 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354926    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-dshm\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:30:42.354989 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354936    2559 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ecdfb810-4349-4404-a1ad-1cf0528f45aa-kserve-provision-location\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:30:42.354989 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354944    2559 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-cache\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:30:42.354989 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.354979    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" (UID: "f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:30:42.355195 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.355173    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" (UID: "f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:30:42.356623 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.356604    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" (UID: "f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:30:42.356701 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.356681    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-kube-api-access-frvff" (OuterVolumeSpecName: "kube-api-access-frvff") pod "f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" (UID: "f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245"). InnerVolumeSpecName "kube-api-access-frvff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:30:42.456336 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.456262    2559 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-uds\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:30:42.456336 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.456290    2559 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tokenizer-tmp\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:30:42.456336 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.456300    2559 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-kserve-provision-location\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:30:42.456336 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.456309    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-tls-certs\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:30:42.456336 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.456318    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frvff\" (UniqueName: \"kubernetes.io/projected/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245-kube-api-access-frvff\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:30:42.901543 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.901504    2559 generic.go:358] "Generic (PLEG): container finished" podID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerID="5adf6718afb477abf3ae73d3773ad5c99a29d8944a326e66811ea87e0e9ae06c" exitCode=0
May 25 13:30:42.902051 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.901589    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" event={"ID":"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245","Type":"ContainerDied","Data":"5adf6718afb477abf3ae73d3773ad5c99a29d8944a326e66811ea87e0e9ae06c"}
May 25 13:30:42.902051 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.901615    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"
May 25 13:30:42.902051 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.901633    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6" event={"ID":"f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245","Type":"ContainerDied","Data":"9238ce2f0f977f78928b2d07383f3127c5693c9f4164b27827965483331d739f"}
May 25 13:30:42.902051 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.901654    2559 scope.go:117] "RemoveContainer" containerID="5adf6718afb477abf3ae73d3773ad5c99a29d8944a326e66811ea87e0e9ae06c"
May 25 13:30:42.903212 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.903189    2559 generic.go:358] "Generic (PLEG): container finished" podID="ecdfb810-4349-4404-a1ad-1cf0528f45aa" containerID="3cbca2e1b874597c1667c8deceb89a4721144bd68e2965acf269507af1e210cd" exitCode=0
May 25 13:30:42.903340 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.903263    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"
May 25 13:30:42.903340 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.903277    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9" event={"ID":"ecdfb810-4349-4404-a1ad-1cf0528f45aa","Type":"ContainerDied","Data":"3cbca2e1b874597c1667c8deceb89a4721144bd68e2965acf269507af1e210cd"}
May 25 13:30:42.903340 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.903314    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9" event={"ID":"ecdfb810-4349-4404-a1ad-1cf0528f45aa","Type":"ContainerDied","Data":"98caafab7e9be2f3e6957e782cf5807479dd7082d00e731e5231823a00db2992"}
May 25 13:30:42.910415 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.910399    2559 scope.go:117] "RemoveContainer" containerID="0c841971456567dd576d9a4058036d134c2b20bcb8e059ddec30a453d26a0695"
May 25 13:30:42.918043 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.918027    2559 scope.go:117] "RemoveContainer" containerID="717932b159e0b9503c76be37484d81015b9c1e4102755d1017bf864773996f97"
May 25 13:30:42.925377 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.925360    2559 scope.go:117] "RemoveContainer" containerID="5adf6718afb477abf3ae73d3773ad5c99a29d8944a326e66811ea87e0e9ae06c"
May 25 13:30:42.925984 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:30:42.925959    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5adf6718afb477abf3ae73d3773ad5c99a29d8944a326e66811ea87e0e9ae06c\": container with ID starting with 5adf6718afb477abf3ae73d3773ad5c99a29d8944a326e66811ea87e0e9ae06c not found: ID does not exist" containerID="5adf6718afb477abf3ae73d3773ad5c99a29d8944a326e66811ea87e0e9ae06c"
May 25 13:30:42.926095 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.925994    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5adf6718afb477abf3ae73d3773ad5c99a29d8944a326e66811ea87e0e9ae06c"} err="failed to get container status \"5adf6718afb477abf3ae73d3773ad5c99a29d8944a326e66811ea87e0e9ae06c\": rpc error: code = NotFound desc = could not find container \"5adf6718afb477abf3ae73d3773ad5c99a29d8944a326e66811ea87e0e9ae06c\": container with ID starting with 5adf6718afb477abf3ae73d3773ad5c99a29d8944a326e66811ea87e0e9ae06c not found: ID does not exist"
May 25 13:30:42.926095 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.926012    2559 scope.go:117] "RemoveContainer" containerID="0c841971456567dd576d9a4058036d134c2b20bcb8e059ddec30a453d26a0695"
May 25 13:30:42.926327 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:30:42.926305    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c841971456567dd576d9a4058036d134c2b20bcb8e059ddec30a453d26a0695\": container with ID starting with 0c841971456567dd576d9a4058036d134c2b20bcb8e059ddec30a453d26a0695 not found: ID does not exist" containerID="0c841971456567dd576d9a4058036d134c2b20bcb8e059ddec30a453d26a0695"
May 25 13:30:42.926400 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.926331    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c841971456567dd576d9a4058036d134c2b20bcb8e059ddec30a453d26a0695"} err="failed to get container status \"0c841971456567dd576d9a4058036d134c2b20bcb8e059ddec30a453d26a0695\": rpc error: code = NotFound desc = could not find container \"0c841971456567dd576d9a4058036d134c2b20bcb8e059ddec30a453d26a0695\": container with ID starting with 0c841971456567dd576d9a4058036d134c2b20bcb8e059ddec30a453d26a0695 not found: ID does not exist"
May 25 13:30:42.926400 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.926359    2559 scope.go:117] "RemoveContainer" containerID="717932b159e0b9503c76be37484d81015b9c1e4102755d1017bf864773996f97"
May 25 13:30:42.926655 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:30:42.926637    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"717932b159e0b9503c76be37484d81015b9c1e4102755d1017bf864773996f97\": container with ID starting with 717932b159e0b9503c76be37484d81015b9c1e4102755d1017bf864773996f97 not found: ID does not exist" containerID="717932b159e0b9503c76be37484d81015b9c1e4102755d1017bf864773996f97"
May 25 13:30:42.926741 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.926659    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"717932b159e0b9503c76be37484d81015b9c1e4102755d1017bf864773996f97"} err="failed to get container status \"717932b159e0b9503c76be37484d81015b9c1e4102755d1017bf864773996f97\": rpc error: code = NotFound desc = could not find container \"717932b159e0b9503c76be37484d81015b9c1e4102755d1017bf864773996f97\": container with ID starting with 717932b159e0b9503c76be37484d81015b9c1e4102755d1017bf864773996f97 not found: ID does not exist"
May 25 13:30:42.926741 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.926676    2559 scope.go:117] "RemoveContainer" containerID="3cbca2e1b874597c1667c8deceb89a4721144bd68e2965acf269507af1e210cd"
May 25 13:30:42.928370 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.928347    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"]
May 25 13:30:42.932426 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.932406    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-855b4f5774-fh4f9"]
May 25 13:30:42.934769 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.934752    2559 scope.go:117] "RemoveContainer" containerID="30a46347929df96b41db4de95af3559283b497ccc5a509eebc2d2f1cb33cd008"
May 25 13:30:42.942692 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.942669    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"]
May 25 13:30:42.948890 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.948871    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7474554d78t6"]
May 25 13:30:42.996586 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.996564    2559 scope.go:117] "RemoveContainer" containerID="3cbca2e1b874597c1667c8deceb89a4721144bd68e2965acf269507af1e210cd"
May 25 13:30:42.996954 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:30:42.996933    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cbca2e1b874597c1667c8deceb89a4721144bd68e2965acf269507af1e210cd\": container with ID starting with 3cbca2e1b874597c1667c8deceb89a4721144bd68e2965acf269507af1e210cd not found: ID does not exist" containerID="3cbca2e1b874597c1667c8deceb89a4721144bd68e2965acf269507af1e210cd"
May 25 13:30:42.997013 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.996961    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cbca2e1b874597c1667c8deceb89a4721144bd68e2965acf269507af1e210cd"} err="failed to get container status \"3cbca2e1b874597c1667c8deceb89a4721144bd68e2965acf269507af1e210cd\": rpc error: code = NotFound desc = could not find container \"3cbca2e1b874597c1667c8deceb89a4721144bd68e2965acf269507af1e210cd\": container with ID starting with 3cbca2e1b874597c1667c8deceb89a4721144bd68e2965acf269507af1e210cd not found: ID does not exist"
May 25 13:30:42.997013 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.996981    2559 scope.go:117] "RemoveContainer" containerID="30a46347929df96b41db4de95af3559283b497ccc5a509eebc2d2f1cb33cd008"
May 25 13:30:42.997246 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:30:42.997220    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30a46347929df96b41db4de95af3559283b497ccc5a509eebc2d2f1cb33cd008\": container with ID starting with 30a46347929df96b41db4de95af3559283b497ccc5a509eebc2d2f1cb33cd008 not found: ID does not exist" containerID="30a46347929df96b41db4de95af3559283b497ccc5a509eebc2d2f1cb33cd008"
May 25 13:30:42.997347 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:42.997246    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30a46347929df96b41db4de95af3559283b497ccc5a509eebc2d2f1cb33cd008"} err="failed to get container status \"30a46347929df96b41db4de95af3559283b497ccc5a509eebc2d2f1cb33cd008\": rpc error: code = NotFound desc = could not find container \"30a46347929df96b41db4de95af3559283b497ccc5a509eebc2d2f1cb33cd008\": container with ID starting with 30a46347929df96b41db4de95af3559283b497ccc5a509eebc2d2f1cb33cd008 not found: ID does not exist"
May 25 13:30:43.869223 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:43.869154    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecdfb810-4349-4404-a1ad-1cf0528f45aa" path="/var/lib/kubelet/pods/ecdfb810-4349-4404-a1ad-1cf0528f45aa/volumes"
May 25 13:30:43.869587 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:30:43.869572    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" path="/var/lib/kubelet/pods/f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245/volumes"
May 25 13:31:02.234118 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234085    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"]
May 25 13:31:02.234736 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234640    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecdfb810-4349-4404-a1ad-1cf0528f45aa" containerName="main"
May 25 13:31:02.234736 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234661    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecdfb810-4349-4404-a1ad-1cf0528f45aa" containerName="main"
May 25 13:31:02.234736 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234690    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main"
May 25 13:31:02.234736 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234698    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main"
May 25 13:31:02.234736 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234713    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecdfb810-4349-4404-a1ad-1cf0528f45aa" containerName="storage-initializer"
May 25 13:31:02.234736 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234737    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecdfb810-4349-4404-a1ad-1cf0528f45aa" containerName="storage-initializer"
May 25 13:31:02.235192 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234752    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="storage-initializer"
May 25 13:31:02.235192 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234761    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="storage-initializer"
May 25 13:31:02.235192 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234774    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="tokenizer"
May 25 13:31:02.235192 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234781    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="tokenizer"
May 25 13:31:02.235192 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234895    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="ecdfb810-4349-4404-a1ad-1cf0528f45aa" containerName="main"
May 25 13:31:02.235192 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234912    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="tokenizer"
May 25 13:31:02.235192 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.234929    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="f54e3aac-fb8a-4c0a-8fcf-b3f7c6a98245" containerName="main"
May 25 13:31:02.238014 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.237991    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.241464 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.241442    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-8blwv\""
May 25 13:31:02.241571 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.241509    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-kserve-self-signed-certs\""
May 25 13:31:02.249303 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.249278    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"]
May 25 13:31:02.330122 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.330090    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-home\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.330122 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.330127    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-tmp-dir\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.330326 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.330160    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4756e927-396b-4fa9-8f4d-09aa688403dc-tls-certs\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.330326 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.330208    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.330326 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.330258    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-dshm\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.330426 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.330338    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-model-cache\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.330426 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.330374    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mdbx\" (UniqueName: \"kubernetes.io/projected/4756e927-396b-4fa9-8f4d-09aa688403dc-kube-api-access-8mdbx\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.431399 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.431363    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-model-cache\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.431588 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.431401    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8mdbx\" (UniqueName: \"kubernetes.io/projected/4756e927-396b-4fa9-8f4d-09aa688403dc-kube-api-access-8mdbx\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.431588 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.431458    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-home\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.431588 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.431486    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-tmp-dir\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.431588 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.431535    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4756e927-396b-4fa9-8f4d-09aa688403dc-tls-certs\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.431588 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.431558    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.431588 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.431583    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-dshm\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.431924 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.431843    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-model-cache\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.431924 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.431876    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-home\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.432038 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.431934    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-tmp-dir\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.432038 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.432008    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.433898 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.433878    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-dshm\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.434155 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.434139    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4756e927-396b-4fa9-8f4d-09aa688403dc-tls-certs\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.439961 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.439938    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mdbx\" (UniqueName: \"kubernetes.io/projected/4756e927-396b-4fa9-8f4d-09aa688403dc-kube-api-access-8mdbx\") pod \"precise-prefix-cache-test-kserve-84f494576-54v5q\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.551409 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.551374    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:31:02.574589 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.574558    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"]
May 25 13:31:02.579580 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.579557    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.582491 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.582461    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"precise-prefix-cache-test-epp-sa-dockercfg-dm2gq\""
May 25 13:31:02.590634 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.590609    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"]
May 25 13:31:02.633738 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.633690    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.633950 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.633876    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.633950 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.633923    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc7s5\" (UniqueName: \"kubernetes.io/projected/b54544a5-d34f-4003-8bc1-94a115dd59c6-kube-api-access-hc7s5\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.634121 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.634015    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b54544a5-d34f-4003-8bc1-94a115dd59c6-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.634121 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.634066    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.634263 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.634127    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.710508 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.705109    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"]
May 25 13:31:02.712163 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:31:02.712132    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4756e927_396b_4fa9_8f4d_09aa688403dc.slice/crio-7d94b315836c70426913d2f681356a604373c6ed1fc6f877d49c2b5e1bdad3d6 WatchSource:0}: Error finding container 7d94b315836c70426913d2f681356a604373c6ed1fc6f877d49c2b5e1bdad3d6: Status 404 returned error can't find the container with id 7d94b315836c70426913d2f681356a604373c6ed1fc6f877d49c2b5e1bdad3d6
May 25 13:31:02.734765 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.734712    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.734878 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.734791    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.734878 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.734813    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hc7s5\" (UniqueName: \"kubernetes.io/projected/b54544a5-d34f-4003-8bc1-94a115dd59c6-kube-api-access-hc7s5\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.734878 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.734835    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b54544a5-d34f-4003-8bc1-94a115dd59c6-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.734878 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.734868    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.735099 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.734908    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.735156 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.735096    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-uds\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.735229 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.735200    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-kserve-provision-location\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.735284 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.735209    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-cache\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.735324 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.735279    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-tmp\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.737432 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.737377    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b54544a5-d34f-4003-8bc1-94a115dd59c6-tls-certs\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.745649 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.745625    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc7s5\" (UniqueName: \"kubernetes.io/projected/b54544a5-d34f-4003-8bc1-94a115dd59c6-kube-api-access-hc7s5\") pod \"precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") " pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.890053 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.889954    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:02.985695 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.985645    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q" event={"ID":"4756e927-396b-4fa9-8f4d-09aa688403dc","Type":"ContainerStarted","Data":"26164a9a3db15780a87a50ae59ba3cc759abdc98b77b927ce60d69ada1521361"}
May 25 13:31:02.985695 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:02.985697    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q" event={"ID":"4756e927-396b-4fa9-8f4d-09aa688403dc","Type":"ContainerStarted","Data":"7d94b315836c70426913d2f681356a604373c6ed1fc6f877d49c2b5e1bdad3d6"}
May 25 13:31:03.039588 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:03.039553    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"]
May 25 13:31:03.043040 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:31:03.043010    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb54544a5_d34f_4003_8bc1_94a115dd59c6.slice/crio-34f7ae88933573115d5f461c47321e73682d8e29b8e0bc241f12721307ab7826 WatchSource:0}: Error finding container 34f7ae88933573115d5f461c47321e73682d8e29b8e0bc241f12721307ab7826: Status 404 returned error can't find the container with id 34f7ae88933573115d5f461c47321e73682d8e29b8e0bc241f12721307ab7826
May 25 13:31:03.991222 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:03.991193    2559 generic.go:358] "Generic (PLEG): container finished" podID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerID="9642f46aa86ed6d9ec58f118fd21c114df2a19e88c6d364aee27c20b6515e7b8" exitCode=0
May 25 13:31:03.991712 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:03.991278    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq" event={"ID":"b54544a5-d34f-4003-8bc1-94a115dd59c6","Type":"ContainerDied","Data":"9642f46aa86ed6d9ec58f118fd21c114df2a19e88c6d364aee27c20b6515e7b8"}
May 25 13:31:03.991712 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:03.991321    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq" event={"ID":"b54544a5-d34f-4003-8bc1-94a115dd59c6","Type":"ContainerStarted","Data":"34f7ae88933573115d5f461c47321e73682d8e29b8e0bc241f12721307ab7826"}
May 25 13:31:04.998164 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:04.998127    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq" event={"ID":"b54544a5-d34f-4003-8bc1-94a115dd59c6","Type":"ContainerStarted","Data":"9d4143ac8e46c05e8960fdf6e7dc46b21a65803d6268a49f160b697905479e99"}
May 25 13:31:04.998164 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:04.998170    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq" event={"ID":"b54544a5-d34f-4003-8bc1-94a115dd59c6","Type":"ContainerStarted","Data":"4e3fad2fdb54ef5e4abdd8c4a34438ed7b3bd07b0408197d7357e27baca56796"}
May 25 13:31:04.998631 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:04.998223    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:05.020749 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:05.020684    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq" podStartSLOduration=3.020669058 podStartE2EDuration="3.020669058s" podCreationTimestamp="2026-05-25 13:31:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:31:05.018179923 +0000 UTC m=+1531.737143087" watchObservedRunningTime="2026-05-25 13:31:05.020669058 +0000 UTC m=+1531.739632196"
May 25 13:31:12.890499 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:12.890442    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:12.890499 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:12.890507    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:12.891795 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:31:12.891769    2559 logging.go:55] [core] [Channel #107 SubChannel #108]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.52:9003", ServerName: "10.133.0.52:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.52:9003: connect: connection refused"
May 25 13:31:12.893047 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:12.893030    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:13.031885 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:13.031854    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:31:13.891304 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:13.891268    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.133.0.52:9003\" within 1s: context deadline exceeded"
May 25 13:31:22.890806 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:31:22.890771    2559 logging.go:55] [core] [Channel #109 SubChannel #110]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.52:9003", ServerName: "10.133.0.52:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.52:9003: connect: connection refused"
May 25 13:31:23.890661 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:23.890615    2559 prober.go:120] "Probe failed" probeType="Liveness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.133.0.52:9003\" within 1s: context deadline exceeded"
May 25 13:31:34.035343 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:31:34.035315    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:32:18.278412 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:32:18.278376    2559 generic.go:358] "Generic (PLEG): container finished" podID="4756e927-396b-4fa9-8f4d-09aa688403dc" containerID="26164a9a3db15780a87a50ae59ba3cc759abdc98b77b927ce60d69ada1521361" exitCode=0
May 25 13:32:18.278853 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:32:18.278450    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q" event={"ID":"4756e927-396b-4fa9-8f4d-09aa688403dc","Type":"ContainerDied","Data":"26164a9a3db15780a87a50ae59ba3cc759abdc98b77b927ce60d69ada1521361"}
May 25 13:32:19.284342 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:32:19.284304    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q" event={"ID":"4756e927-396b-4fa9-8f4d-09aa688403dc","Type":"ContainerStarted","Data":"ac138565d5f8ec277bd11b827cbdc990168b896a4a11171e4045916076ee2c20"}
May 25 13:32:19.304922 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:32:19.304869    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q" podStartSLOduration=77.304852118 podStartE2EDuration="1m17.304852118s" podCreationTimestamp="2026-05-25 13:31:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:32:19.302710147 +0000 UTC m=+1606.021673287" watchObservedRunningTime="2026-05-25 13:32:19.304852118 +0000 UTC m=+1606.023815260"
May 25 13:32:22.551498 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:32:22.551459    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:32:22.551948 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:32:22.551610    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:32:22.564263 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:32:22.564236    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:32:23.310512 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:32:23.310485    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:45:43.474764 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.474700    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"]
May 25 13:45:43.475367 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.474783    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"]
May 25 13:45:43.475367 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.475196    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q" podUID="4756e927-396b-4fa9-8f4d-09aa688403dc" containerName="main" containerID="cri-o://ac138565d5f8ec277bd11b827cbdc990168b896a4a11171e4045916076ee2c20" gracePeriod=30
May 25 13:45:43.476055 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.475744    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerName="tokenizer" containerID="cri-o://9d4143ac8e46c05e8960fdf6e7dc46b21a65803d6268a49f160b697905479e99" gracePeriod=30
May 25 13:45:43.476055 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.475754    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerName="main" containerID="cri-o://4e3fad2fdb54ef5e4abdd8c4a34438ed7b3bd07b0408197d7357e27baca56796" gracePeriod=30
May 25 13:45:43.732033 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.731966    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:45:43.749566 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.749537    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-model-cache\") pod \"4756e927-396b-4fa9-8f4d-09aa688403dc\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") "
May 25 13:45:43.749709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.749589    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-tmp-dir\") pod \"4756e927-396b-4fa9-8f4d-09aa688403dc\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") "
May 25 13:45:43.749709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.749614    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-dshm\") pod \"4756e927-396b-4fa9-8f4d-09aa688403dc\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") "
May 25 13:45:43.749709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.749636    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mdbx\" (UniqueName: \"kubernetes.io/projected/4756e927-396b-4fa9-8f4d-09aa688403dc-kube-api-access-8mdbx\") pod \"4756e927-396b-4fa9-8f4d-09aa688403dc\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") "
May 25 13:45:43.749709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.749655    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-home\") pod \"4756e927-396b-4fa9-8f4d-09aa688403dc\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") "
May 25 13:45:43.749709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.749704    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4756e927-396b-4fa9-8f4d-09aa688403dc-tls-certs\") pod \"4756e927-396b-4fa9-8f4d-09aa688403dc\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") "
May 25 13:45:43.749987 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.749757    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-kserve-provision-location\") pod \"4756e927-396b-4fa9-8f4d-09aa688403dc\" (UID: \"4756e927-396b-4fa9-8f4d-09aa688403dc\") "
May 25 13:45:43.749987 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.749795    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "4756e927-396b-4fa9-8f4d-09aa688403dc" (UID: "4756e927-396b-4fa9-8f4d-09aa688403dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:45:43.749987 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.749925    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4756e927-396b-4fa9-8f4d-09aa688403dc" (UID: "4756e927-396b-4fa9-8f4d-09aa688403dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:45:43.750136 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.750025    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-home" (OuterVolumeSpecName: "home") pod "4756e927-396b-4fa9-8f4d-09aa688403dc" (UID: "4756e927-396b-4fa9-8f4d-09aa688403dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:45:43.750136 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.750128    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-model-cache\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:45:43.750329 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.750148    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-tmp-dir\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:45:43.750329 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.750161    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-home\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:45:43.751980 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.751936    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4756e927-396b-4fa9-8f4d-09aa688403dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4756e927-396b-4fa9-8f4d-09aa688403dc" (UID: "4756e927-396b-4fa9-8f4d-09aa688403dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:45:43.751980 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.751939    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4756e927-396b-4fa9-8f4d-09aa688403dc-kube-api-access-8mdbx" (OuterVolumeSpecName: "kube-api-access-8mdbx") pod "4756e927-396b-4fa9-8f4d-09aa688403dc" (UID: "4756e927-396b-4fa9-8f4d-09aa688403dc"). InnerVolumeSpecName "kube-api-access-8mdbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:45:43.752244 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.752220    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-dshm" (OuterVolumeSpecName: "dshm") pod "4756e927-396b-4fa9-8f4d-09aa688403dc" (UID: "4756e927-396b-4fa9-8f4d-09aa688403dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:45:43.811412 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.811363    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4756e927-396b-4fa9-8f4d-09aa688403dc" (UID: "4756e927-396b-4fa9-8f4d-09aa688403dc"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:45:43.851668 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.851592    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-dshm\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:45:43.851668 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.851629    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mdbx\" (UniqueName: \"kubernetes.io/projected/4756e927-396b-4fa9-8f4d-09aa688403dc-kube-api-access-8mdbx\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:45:43.851668 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.851645    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4756e927-396b-4fa9-8f4d-09aa688403dc-tls-certs\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:45:43.851668 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:43.851659    2559 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4756e927-396b-4fa9-8f4d-09aa688403dc-kserve-provision-location\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:45:44.035546 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:45:44.035515    2559 logging.go:55] [core] [Channel #624 SubChannel #625]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.52:9003", ServerName: "10.133.0.52:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.52:9003: connect: connection refused"
May 25 13:45:44.338617 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.338532    2559 generic.go:358] "Generic (PLEG): container finished" podID="4756e927-396b-4fa9-8f4d-09aa688403dc" containerID="ac138565d5f8ec277bd11b827cbdc990168b896a4a11171e4045916076ee2c20" exitCode=0
May 25 13:45:44.338801 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.338616    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q" event={"ID":"4756e927-396b-4fa9-8f4d-09aa688403dc","Type":"ContainerDied","Data":"ac138565d5f8ec277bd11b827cbdc990168b896a4a11171e4045916076ee2c20"}
May 25 13:45:44.338801 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.338649    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q" event={"ID":"4756e927-396b-4fa9-8f4d-09aa688403dc","Type":"ContainerDied","Data":"7d94b315836c70426913d2f681356a604373c6ed1fc6f877d49c2b5e1bdad3d6"}
May 25 13:45:44.338801 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.338668    2559 scope.go:117] "RemoveContainer" containerID="ac138565d5f8ec277bd11b827cbdc990168b896a4a11171e4045916076ee2c20"
May 25 13:45:44.338801 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.338624    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"
May 25 13:45:44.341546 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.341521    2559 generic.go:358] "Generic (PLEG): container finished" podID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerID="4e3fad2fdb54ef5e4abdd8c4a34438ed7b3bd07b0408197d7357e27baca56796" exitCode=0
May 25 13:45:44.341646 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.341589    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq" event={"ID":"b54544a5-d34f-4003-8bc1-94a115dd59c6","Type":"ContainerDied","Data":"4e3fad2fdb54ef5e4abdd8c4a34438ed7b3bd07b0408197d7357e27baca56796"}
May 25 13:45:44.349407 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.349377    2559 scope.go:117] "RemoveContainer" containerID="26164a9a3db15780a87a50ae59ba3cc759abdc98b77b927ce60d69ada1521361"
May 25 13:45:44.357482 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.357454    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"]
May 25 13:45:44.362295 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.362268    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-84f494576-54v5q"]
May 25 13:45:44.412872 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.412847    2559 scope.go:117] "RemoveContainer" containerID="ac138565d5f8ec277bd11b827cbdc990168b896a4a11171e4045916076ee2c20"
May 25 13:45:44.413233 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:45:44.413212    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac138565d5f8ec277bd11b827cbdc990168b896a4a11171e4045916076ee2c20\": container with ID starting with ac138565d5f8ec277bd11b827cbdc990168b896a4a11171e4045916076ee2c20 not found: ID does not exist" containerID="ac138565d5f8ec277bd11b827cbdc990168b896a4a11171e4045916076ee2c20"
May 25 13:45:44.413297 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.413241    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac138565d5f8ec277bd11b827cbdc990168b896a4a11171e4045916076ee2c20"} err="failed to get container status \"ac138565d5f8ec277bd11b827cbdc990168b896a4a11171e4045916076ee2c20\": rpc error: code = NotFound desc = could not find container \"ac138565d5f8ec277bd11b827cbdc990168b896a4a11171e4045916076ee2c20\": container with ID starting with ac138565d5f8ec277bd11b827cbdc990168b896a4a11171e4045916076ee2c20 not found: ID does not exist"
May 25 13:45:44.413297 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.413263    2559 scope.go:117] "RemoveContainer" containerID="26164a9a3db15780a87a50ae59ba3cc759abdc98b77b927ce60d69ada1521361"
May 25 13:45:44.413533 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:45:44.413515    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26164a9a3db15780a87a50ae59ba3cc759abdc98b77b927ce60d69ada1521361\": container with ID starting with 26164a9a3db15780a87a50ae59ba3cc759abdc98b77b927ce60d69ada1521361 not found: ID does not exist" containerID="26164a9a3db15780a87a50ae59ba3cc759abdc98b77b927ce60d69ada1521361"
May 25 13:45:44.413581 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:44.413540    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26164a9a3db15780a87a50ae59ba3cc759abdc98b77b927ce60d69ada1521361"} err="failed to get container status \"26164a9a3db15780a87a50ae59ba3cc759abdc98b77b927ce60d69ada1521361\": rpc error: code = NotFound desc = could not find container \"26164a9a3db15780a87a50ae59ba3cc759abdc98b77b927ce60d69ada1521361\": container with ID starting with 26164a9a3db15780a87a50ae59ba3cc759abdc98b77b927ce60d69ada1521361 not found: ID does not exist"
May 25 13:45:45.028667 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.028644    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:45:45.035865 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.035837    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.133.0.52:9003\" within 1s: context deadline exceeded"
May 25 13:45:45.063165 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.063080    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-kserve-provision-location\") pod \"b54544a5-d34f-4003-8bc1-94a115dd59c6\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") "
May 25 13:45:45.063165 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.063127    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-uds\") pod \"b54544a5-d34f-4003-8bc1-94a115dd59c6\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") "
May 25 13:45:45.063370 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.063193    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b54544a5-d34f-4003-8bc1-94a115dd59c6-tls-certs\") pod \"b54544a5-d34f-4003-8bc1-94a115dd59c6\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") "
May 25 13:45:45.063370 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.063212    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-tmp\") pod \"b54544a5-d34f-4003-8bc1-94a115dd59c6\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") "
May 25 13:45:45.063370 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.063251    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-cache\") pod \"b54544a5-d34f-4003-8bc1-94a115dd59c6\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") "
May 25 13:45:45.063370 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.063283    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc7s5\" (UniqueName: \"kubernetes.io/projected/b54544a5-d34f-4003-8bc1-94a115dd59c6-kube-api-access-hc7s5\") pod \"b54544a5-d34f-4003-8bc1-94a115dd59c6\" (UID: \"b54544a5-d34f-4003-8bc1-94a115dd59c6\") "
May 25 13:45:45.063582 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.063430    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "b54544a5-d34f-4003-8bc1-94a115dd59c6" (UID: "b54544a5-d34f-4003-8bc1-94a115dd59c6"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:45:45.063582 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.063558    2559 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-uds\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:45:45.063689 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.063620    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "b54544a5-d34f-4003-8bc1-94a115dd59c6" (UID: "b54544a5-d34f-4003-8bc1-94a115dd59c6"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:45:45.063689 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.063641    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "b54544a5-d34f-4003-8bc1-94a115dd59c6" (UID: "b54544a5-d34f-4003-8bc1-94a115dd59c6"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:45:45.063973 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.063951    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b54544a5-d34f-4003-8bc1-94a115dd59c6" (UID: "b54544a5-d34f-4003-8bc1-94a115dd59c6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:45:45.065177 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.065160    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b54544a5-d34f-4003-8bc1-94a115dd59c6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b54544a5-d34f-4003-8bc1-94a115dd59c6" (UID: "b54544a5-d34f-4003-8bc1-94a115dd59c6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:45:45.065338 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.065316    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b54544a5-d34f-4003-8bc1-94a115dd59c6-kube-api-access-hc7s5" (OuterVolumeSpecName: "kube-api-access-hc7s5") pod "b54544a5-d34f-4003-8bc1-94a115dd59c6" (UID: "b54544a5-d34f-4003-8bc1-94a115dd59c6"). InnerVolumeSpecName "kube-api-access-hc7s5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:45:45.164700 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.164672    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc7s5\" (UniqueName: \"kubernetes.io/projected/b54544a5-d34f-4003-8bc1-94a115dd59c6-kube-api-access-hc7s5\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:45:45.164700 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.164699    2559 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-kserve-provision-location\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:45:45.164898 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.164709    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b54544a5-d34f-4003-8bc1-94a115dd59c6-tls-certs\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:45:45.164898 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.164732    2559 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-tmp\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:45:45.164898 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.164742    2559 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/b54544a5-d34f-4003-8bc1-94a115dd59c6-tokenizer-cache\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:45:45.348605 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.348521    2559 generic.go:358] "Generic (PLEG): container finished" podID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerID="9d4143ac8e46c05e8960fdf6e7dc46b21a65803d6268a49f160b697905479e99" exitCode=0
May 25 13:45:45.348605 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.348575    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq" event={"ID":"b54544a5-d34f-4003-8bc1-94a115dd59c6","Type":"ContainerDied","Data":"9d4143ac8e46c05e8960fdf6e7dc46b21a65803d6268a49f160b697905479e99"}
May 25 13:45:45.348605 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.348604    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"
May 25 13:45:45.348842 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.348618    2559 scope.go:117] "RemoveContainer" containerID="9d4143ac8e46c05e8960fdf6e7dc46b21a65803d6268a49f160b697905479e99"
May 25 13:45:45.348842 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.348606    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq" event={"ID":"b54544a5-d34f-4003-8bc1-94a115dd59c6","Type":"ContainerDied","Data":"34f7ae88933573115d5f461c47321e73682d8e29b8e0bc241f12721307ab7826"}
May 25 13:45:45.357178 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.357160    2559 scope.go:117] "RemoveContainer" containerID="4e3fad2fdb54ef5e4abdd8c4a34438ed7b3bd07b0408197d7357e27baca56796"
May 25 13:45:45.364967 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.364947    2559 scope.go:117] "RemoveContainer" containerID="9642f46aa86ed6d9ec58f118fd21c114df2a19e88c6d364aee27c20b6515e7b8"
May 25 13:45:45.371290 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.371267    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"]
May 25 13:45:45.374954 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.374936    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-858b6cd9c4xdq"]
May 25 13:45:45.375029 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.375001    2559 scope.go:117] "RemoveContainer" containerID="9d4143ac8e46c05e8960fdf6e7dc46b21a65803d6268a49f160b697905479e99"
May 25 13:45:45.375293 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:45:45.375275    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d4143ac8e46c05e8960fdf6e7dc46b21a65803d6268a49f160b697905479e99\": container with ID starting with 9d4143ac8e46c05e8960fdf6e7dc46b21a65803d6268a49f160b697905479e99 not found: ID does not exist" containerID="9d4143ac8e46c05e8960fdf6e7dc46b21a65803d6268a49f160b697905479e99"
May 25 13:45:45.375361 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.375305    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d4143ac8e46c05e8960fdf6e7dc46b21a65803d6268a49f160b697905479e99"} err="failed to get container status \"9d4143ac8e46c05e8960fdf6e7dc46b21a65803d6268a49f160b697905479e99\": rpc error: code = NotFound desc = could not find container \"9d4143ac8e46c05e8960fdf6e7dc46b21a65803d6268a49f160b697905479e99\": container with ID starting with 9d4143ac8e46c05e8960fdf6e7dc46b21a65803d6268a49f160b697905479e99 not found: ID does not exist"
May 25 13:45:45.375361 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.375334    2559 scope.go:117] "RemoveContainer" containerID="4e3fad2fdb54ef5e4abdd8c4a34438ed7b3bd07b0408197d7357e27baca56796"
May 25 13:45:45.375594 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:45:45.375579    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e3fad2fdb54ef5e4abdd8c4a34438ed7b3bd07b0408197d7357e27baca56796\": container with ID starting with 4e3fad2fdb54ef5e4abdd8c4a34438ed7b3bd07b0408197d7357e27baca56796 not found: ID does not exist" containerID="4e3fad2fdb54ef5e4abdd8c4a34438ed7b3bd07b0408197d7357e27baca56796"
May 25 13:45:45.375636 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.375600    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e3fad2fdb54ef5e4abdd8c4a34438ed7b3bd07b0408197d7357e27baca56796"} err="failed to get container status \"4e3fad2fdb54ef5e4abdd8c4a34438ed7b3bd07b0408197d7357e27baca56796\": rpc error: code = NotFound desc = could not find container \"4e3fad2fdb54ef5e4abdd8c4a34438ed7b3bd07b0408197d7357e27baca56796\": container with ID starting with 4e3fad2fdb54ef5e4abdd8c4a34438ed7b3bd07b0408197d7357e27baca56796 not found: ID does not exist"
May 25 13:45:45.375636 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.375615    2559 scope.go:117] "RemoveContainer" containerID="9642f46aa86ed6d9ec58f118fd21c114df2a19e88c6d364aee27c20b6515e7b8"
May 25 13:45:45.375868 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:45:45.375848    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9642f46aa86ed6d9ec58f118fd21c114df2a19e88c6d364aee27c20b6515e7b8\": container with ID starting with 9642f46aa86ed6d9ec58f118fd21c114df2a19e88c6d364aee27c20b6515e7b8 not found: ID does not exist" containerID="9642f46aa86ed6d9ec58f118fd21c114df2a19e88c6d364aee27c20b6515e7b8"
May 25 13:45:45.375922 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.375873    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9642f46aa86ed6d9ec58f118fd21c114df2a19e88c6d364aee27c20b6515e7b8"} err="failed to get container status \"9642f46aa86ed6d9ec58f118fd21c114df2a19e88c6d364aee27c20b6515e7b8\": rpc error: code = NotFound desc = could not find container \"9642f46aa86ed6d9ec58f118fd21c114df2a19e88c6d364aee27c20b6515e7b8\": container with ID starting with 9642f46aa86ed6d9ec58f118fd21c114df2a19e88c6d364aee27c20b6515e7b8 not found: ID does not exist"
May 25 13:45:45.869894 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.869863    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4756e927-396b-4fa9-8f4d-09aa688403dc" path="/var/lib/kubelet/pods/4756e927-396b-4fa9-8f4d-09aa688403dc/volumes"
May 25 13:45:45.870299 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:45:45.870287    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" path="/var/lib/kubelet/pods/b54544a5-d34f-4003-8bc1-94a115dd59c6/volumes"
May 25 13:46:04.162447 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.162410    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"]
May 25 13:46:04.162944 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.162925    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerName="main"
May 25 13:46:04.163025 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.162948    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerName="main"
May 25 13:46:04.163025 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.162965    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerName="storage-initializer"
May 25 13:46:04.163025 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.162974    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerName="storage-initializer"
May 25 13:46:04.163025 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.162990    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerName="tokenizer"
May 25 13:46:04.163025 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.162999    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerName="tokenizer"
May 25 13:46:04.163025 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.163008    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4756e927-396b-4fa9-8f4d-09aa688403dc" containerName="main"
May 25 13:46:04.163025 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.163016    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4756e927-396b-4fa9-8f4d-09aa688403dc" containerName="main"
May 25 13:46:04.163025 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.163025    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4756e927-396b-4fa9-8f4d-09aa688403dc" containerName="storage-initializer"
May 25 13:46:04.163417 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.163032    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4756e927-396b-4fa9-8f4d-09aa688403dc" containerName="storage-initializer"
May 25 13:46:04.163417 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.163129    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerName="tokenizer"
May 25 13:46:04.163417 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.163145    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="4756e927-396b-4fa9-8f4d-09aa688403dc" containerName="main"
May 25 13:46:04.163417 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.163156    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="b54544a5-d34f-4003-8bc1-94a115dd59c6" containerName="main"
May 25 13:46:04.165393 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.165373    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.169178 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.169148    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-kserve-self-signed-certs\""
May 25 13:46:04.170061 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.170002    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-8blwv\""
May 25 13:46:04.170191 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.170015    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-epp-sa-dockercfg-6gh56\""
May 25 13:46:04.178129 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.178096    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"]
May 25 13:46:04.225496 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.225449    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.225673 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.225581    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.225673 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.225617    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jjh7\" (UniqueName: \"kubernetes.io/projected/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-kube-api-access-9jjh7\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.225673 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.225652    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.225844 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.225763    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.225844 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.225793    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.326691 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.326655    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.326691 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.326693    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.326957 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.326751    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.326957 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.326821    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.326957 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.326848    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9jjh7\" (UniqueName: \"kubernetes.io/projected/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-kube-api-access-9jjh7\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.326957 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.326882    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.327233 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.327136    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.327233 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.327182    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.327318 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.327234    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.327353 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.327336    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.329359 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.329333    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.334663 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.334640    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jjh7\" (UniqueName: \"kubernetes.io/projected/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-kube-api-access-9jjh7\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.483650 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.483564    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:04.621640 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.621612    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"]
May 25 13:46:04.623995 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:46:04.623960    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c13026_b7cd_4616_b94d_84b47b8cdd1a.slice/crio-17d0fff8dd2c2f48efb1adbddb0b5629033966a57214c201c611c65ea9ac5c0c WatchSource:0}: Error finding container 17d0fff8dd2c2f48efb1adbddb0b5629033966a57214c201c611c65ea9ac5c0c: Status 404 returned error can't find the container with id 17d0fff8dd2c2f48efb1adbddb0b5629033966a57214c201c611c65ea9ac5c0c
May 25 13:46:04.625989 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:04.625970    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 13:46:05.428958 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:05.428925    2559 generic.go:358] "Generic (PLEG): container finished" podID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerID="ccc22c8d2c8253ec7fdd5a16e5568f66e75da7974ce377eb67f373b647d518fc" exitCode=0
May 25 13:46:05.429425 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:05.428987    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd" event={"ID":"c1c13026-b7cd-4616-b94d-84b47b8cdd1a","Type":"ContainerDied","Data":"ccc22c8d2c8253ec7fdd5a16e5568f66e75da7974ce377eb67f373b647d518fc"}
May 25 13:46:05.429425 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:05.429011    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd" event={"ID":"c1c13026-b7cd-4616-b94d-84b47b8cdd1a","Type":"ContainerStarted","Data":"17d0fff8dd2c2f48efb1adbddb0b5629033966a57214c201c611c65ea9ac5c0c"}
May 25 13:46:06.435706 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:06.435675    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd" event={"ID":"c1c13026-b7cd-4616-b94d-84b47b8cdd1a","Type":"ContainerStarted","Data":"67e76b8210fc45441f56da22c6e9c23bc46e47a011a0f3e6c6eaaa58adae1461"}
May 25 13:46:06.435706 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:06.435711    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd" event={"ID":"c1c13026-b7cd-4616-b94d-84b47b8cdd1a","Type":"ContainerStarted","Data":"6c021f9a81083bc93c700937c5a1d342183d65bd3ee39884806dee089c24a678"}
May 25 13:46:06.436142 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:06.435839    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:06.460949 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:06.460903    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd" podStartSLOduration=2.460889903 podStartE2EDuration="2.460889903s" podCreationTimestamp="2026-05-25 13:46:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:46:06.45788935 +0000 UTC m=+2433.176852491" watchObservedRunningTime="2026-05-25 13:46:06.460889903 +0000 UTC m=+2433.179853081"
May 25 13:46:14.483783 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:14.483743    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:14.483783 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:14.483792    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:14.486352 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:14.486329    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:15.473054 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:15.473028    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:46:36.478157 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:46:36.478121    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:47:45.384219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:45.384124    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"]
May 25 13:47:45.384743 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:45.384536    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd" podUID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerName="main" containerID="cri-o://6c021f9a81083bc93c700937c5a1d342183d65bd3ee39884806dee089c24a678" gracePeriod=30
May 25 13:47:45.384743 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:45.384569    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd" podUID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerName="tokenizer" containerID="cri-o://67e76b8210fc45441f56da22c6e9c23bc46e47a011a0f3e6c6eaaa58adae1461" gracePeriod=30
May 25 13:47:45.472897 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:45.472850    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd" podUID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerName="tokenizer" probeResult="failure" output="Get \"http://10.133.0.53:8082/healthz\": dial tcp 10.133.0.53:8082: connect: connection refused"
May 25 13:47:45.819234 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:45.819199    2559 generic.go:358] "Generic (PLEG): container finished" podID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerID="6c021f9a81083bc93c700937c5a1d342183d65bd3ee39884806dee089c24a678" exitCode=0
May 25 13:47:45.819412 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:45.819271    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd" event={"ID":"c1c13026-b7cd-4616-b94d-84b47b8cdd1a","Type":"ContainerDied","Data":"6c021f9a81083bc93c700937c5a1d342183d65bd3ee39884806dee089c24a678"}
May 25 13:47:46.476933 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:47:46.476902    2559 logging.go:55] [core] [Channel #677 SubChannel #678]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.53:9003", ServerName: "10.133.0.53:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.53:9003: connect: connection refused"
May 25 13:47:46.637317 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.637295    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:47:46.727981 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.727893    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jjh7\" (UniqueName: \"kubernetes.io/projected/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-kube-api-access-9jjh7\") pod \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") "
May 25 13:47:46.727981 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.727929    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-tmp\") pod \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") "
May 25 13:47:46.727981 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.727976    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-uds\") pod \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") "
May 25 13:47:46.728227 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.727997    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tls-certs\") pod \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") "
May 25 13:47:46.728227 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.728018    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-cache\") pod \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") "
May 25 13:47:46.728227 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.728046    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-kserve-provision-location\") pod \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\" (UID: \"c1c13026-b7cd-4616-b94d-84b47b8cdd1a\") "
May 25 13:47:46.728362 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.728227    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "c1c13026-b7cd-4616-b94d-84b47b8cdd1a" (UID: "c1c13026-b7cd-4616-b94d-84b47b8cdd1a"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:47:46.728362 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.728301    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "c1c13026-b7cd-4616-b94d-84b47b8cdd1a" (UID: "c1c13026-b7cd-4616-b94d-84b47b8cdd1a"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:47:46.731068 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.728793    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "c1c13026-b7cd-4616-b94d-84b47b8cdd1a" (UID: "c1c13026-b7cd-4616-b94d-84b47b8cdd1a"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:47:46.731068 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.729367    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c1c13026-b7cd-4616-b94d-84b47b8cdd1a" (UID: "c1c13026-b7cd-4616-b94d-84b47b8cdd1a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:47:46.735952 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.735928    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-kube-api-access-9jjh7" (OuterVolumeSpecName: "kube-api-access-9jjh7") pod "c1c13026-b7cd-4616-b94d-84b47b8cdd1a" (UID: "c1c13026-b7cd-4616-b94d-84b47b8cdd1a"). InnerVolumeSpecName "kube-api-access-9jjh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:47:46.736055 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.735943    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c1c13026-b7cd-4616-b94d-84b47b8cdd1a" (UID: "c1c13026-b7cd-4616-b94d-84b47b8cdd1a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:47:46.824641 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.824607    2559 generic.go:358] "Generic (PLEG): container finished" podID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerID="67e76b8210fc45441f56da22c6e9c23bc46e47a011a0f3e6c6eaaa58adae1461" exitCode=0
May 25 13:47:46.824843 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.824706    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"
May 25 13:47:46.824843 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.824698    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd" event={"ID":"c1c13026-b7cd-4616-b94d-84b47b8cdd1a","Type":"ContainerDied","Data":"67e76b8210fc45441f56da22c6e9c23bc46e47a011a0f3e6c6eaaa58adae1461"}
May 25 13:47:46.824966 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.824851    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd" event={"ID":"c1c13026-b7cd-4616-b94d-84b47b8cdd1a","Type":"ContainerDied","Data":"17d0fff8dd2c2f48efb1adbddb0b5629033966a57214c201c611c65ea9ac5c0c"}
May 25 13:47:46.824966 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.824877    2559 scope.go:117] "RemoveContainer" containerID="67e76b8210fc45441f56da22c6e9c23bc46e47a011a0f3e6c6eaaa58adae1461"
May 25 13:47:46.828884 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.828859    2559 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-uds\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:47:46.828884 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.828883    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tls-certs\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:47:46.829061 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.828892    2559 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-cache\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:47:46.829061 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.828903    2559 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-kserve-provision-location\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:47:46.829061 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.828913    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jjh7\" (UniqueName: \"kubernetes.io/projected/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-kube-api-access-9jjh7\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:47:46.829061 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.828921    2559 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/c1c13026-b7cd-4616-b94d-84b47b8cdd1a-tokenizer-tmp\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:47:46.833709 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.833692    2559 scope.go:117] "RemoveContainer" containerID="6c021f9a81083bc93c700937c5a1d342183d65bd3ee39884806dee089c24a678"
May 25 13:47:46.841285 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.841270    2559 scope.go:117] "RemoveContainer" containerID="ccc22c8d2c8253ec7fdd5a16e5568f66e75da7974ce377eb67f373b647d518fc"
May 25 13:47:46.848515 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.848494    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"]
May 25 13:47:46.849680 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.849666    2559 scope.go:117] "RemoveContainer" containerID="67e76b8210fc45441f56da22c6e9c23bc46e47a011a0f3e6c6eaaa58adae1461"
May 25 13:47:46.849940 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:47:46.849913    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67e76b8210fc45441f56da22c6e9c23bc46e47a011a0f3e6c6eaaa58adae1461\": container with ID starting with 67e76b8210fc45441f56da22c6e9c23bc46e47a011a0f3e6c6eaaa58adae1461 not found: ID does not exist" containerID="67e76b8210fc45441f56da22c6e9c23bc46e47a011a0f3e6c6eaaa58adae1461"
May 25 13:47:46.850008 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.849941    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67e76b8210fc45441f56da22c6e9c23bc46e47a011a0f3e6c6eaaa58adae1461"} err="failed to get container status \"67e76b8210fc45441f56da22c6e9c23bc46e47a011a0f3e6c6eaaa58adae1461\": rpc error: code = NotFound desc = could not find container \"67e76b8210fc45441f56da22c6e9c23bc46e47a011a0f3e6c6eaaa58adae1461\": container with ID starting with 67e76b8210fc45441f56da22c6e9c23bc46e47a011a0f3e6c6eaaa58adae1461 not found: ID does not exist"
May 25 13:47:46.850008 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.849960    2559 scope.go:117] "RemoveContainer" containerID="6c021f9a81083bc93c700937c5a1d342183d65bd3ee39884806dee089c24a678"
May 25 13:47:46.850191 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:47:46.850172    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c021f9a81083bc93c700937c5a1d342183d65bd3ee39884806dee089c24a678\": container with ID starting with 6c021f9a81083bc93c700937c5a1d342183d65bd3ee39884806dee089c24a678 not found: ID does not exist" containerID="6c021f9a81083bc93c700937c5a1d342183d65bd3ee39884806dee089c24a678"
May 25 13:47:46.850238 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.850196    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c021f9a81083bc93c700937c5a1d342183d65bd3ee39884806dee089c24a678"} err="failed to get container status \"6c021f9a81083bc93c700937c5a1d342183d65bd3ee39884806dee089c24a678\": rpc error: code = NotFound desc = could not find container \"6c021f9a81083bc93c700937c5a1d342183d65bd3ee39884806dee089c24a678\": container with ID starting with 6c021f9a81083bc93c700937c5a1d342183d65bd3ee39884806dee089c24a678 not found: ID does not exist"
May 25 13:47:46.850238 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.850210    2559 scope.go:117] "RemoveContainer" containerID="ccc22c8d2c8253ec7fdd5a16e5568f66e75da7974ce377eb67f373b647d518fc"
May 25 13:47:46.850407 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:47:46.850391    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccc22c8d2c8253ec7fdd5a16e5568f66e75da7974ce377eb67f373b647d518fc\": container with ID starting with ccc22c8d2c8253ec7fdd5a16e5568f66e75da7974ce377eb67f373b647d518fc not found: ID does not exist" containerID="ccc22c8d2c8253ec7fdd5a16e5568f66e75da7974ce377eb67f373b647d518fc"
May 25 13:47:46.850452 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.850411    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc22c8d2c8253ec7fdd5a16e5568f66e75da7974ce377eb67f373b647d518fc"} err="failed to get container status \"ccc22c8d2c8253ec7fdd5a16e5568f66e75da7974ce377eb67f373b647d518fc\": rpc error: code = NotFound desc = could not find container \"ccc22c8d2c8253ec7fdd5a16e5568f66e75da7974ce377eb67f373b647d518fc\": container with ID starting with ccc22c8d2c8253ec7fdd5a16e5568f66e75da7974ce377eb67f373b647d518fc not found: ID does not exist"
May 25 13:47:46.854356 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:46.854335    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd"]
May 25 13:47:47.476574 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:47.476525    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-q9vbd" podUID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.133.0.53:9003\" within 1s: context deadline exceeded"
May 25 13:47:47.870175 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:47:47.870141    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" path="/var/lib/kubelet/pods/c1c13026-b7cd-4616-b94d-84b47b8cdd1a/volumes"
May 25 13:48:08.269332 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.269298    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"]
May 25 13:48:08.269785 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.269650    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerName="main"
May 25 13:48:08.269785 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.269663    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerName="main"
May 25 13:48:08.269785 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.269682    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerName="storage-initializer"
May 25 13:48:08.269785 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.269688    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerName="storage-initializer"
May 25 13:48:08.269785 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.269698    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerName="tokenizer"
May 25 13:48:08.269785 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.269704    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerName="tokenizer"
May 25 13:48:08.269991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.269799    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerName="tokenizer"
May 25 13:48:08.269991 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.269809    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="c1c13026-b7cd-4616-b94d-84b47b8cdd1a" containerName="main"
May 25 13:48:08.273010 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.272994    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.275340 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.275315    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-8blwv\""
May 25 13:48:08.275936 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.275918    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-epp-sa-dockercfg-bcvl4\""
May 25 13:48:08.276004 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.275941    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"stop-feature-test-kserve-self-signed-certs\""
May 25 13:48:08.284317 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.284291    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"]
May 25 13:48:08.314545 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.314504    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g78fl\" (UniqueName: \"kubernetes.io/projected/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-kube-api-access-g78fl\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.314702 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.314558    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.314702 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.314615    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.314814 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.314691    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.314814 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.314762    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.314814 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.314806    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.415339 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.415290    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.415339 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.415343    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.415568 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.415372    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.415568 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.415413    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g78fl\" (UniqueName: \"kubernetes.io/projected/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-kube-api-access-g78fl\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.415568 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.415467    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.415568 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.415515    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.415771 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.415748    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-tmp\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.415816 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.415798    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-cache\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.415854 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.415830    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-uds\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.415890 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.415864    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-kserve-provision-location\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.417921 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.417898    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.423419 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.423394    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g78fl\" (UniqueName: \"kubernetes.io/projected/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-kube-api-access-g78fl\") pod \"stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") " pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.583965 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.583868    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:08.719894 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.719865    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"]
May 25 13:48:08.721984 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:48:08.721949    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebc2ca11_e6eb_4eda_94e3_971c863ce1ae.slice/crio-c5acf1dd1532c4881f8d0e0b304a654f9f089f0e7bda6720916b14680c8f28cd WatchSource:0}: Error finding container c5acf1dd1532c4881f8d0e0b304a654f9f089f0e7bda6720916b14680c8f28cd: Status 404 returned error can't find the container with id c5acf1dd1532c4881f8d0e0b304a654f9f089f0e7bda6720916b14680c8f28cd
May 25 13:48:08.908403 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.908313    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg" event={"ID":"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae","Type":"ContainerStarted","Data":"713a72f43a32066cc0936db71c0d431afd1ff01fa8b3fcf3f6c7328ef31bcc8c"}
May 25 13:48:08.908403 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:08.908364    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg" event={"ID":"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae","Type":"ContainerStarted","Data":"c5acf1dd1532c4881f8d0e0b304a654f9f089f0e7bda6720916b14680c8f28cd"}
May 25 13:48:09.914255 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:09.914219    2559 generic.go:358] "Generic (PLEG): container finished" podID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerID="713a72f43a32066cc0936db71c0d431afd1ff01fa8b3fcf3f6c7328ef31bcc8c" exitCode=0
May 25 13:48:09.914763 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:09.914273    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg" event={"ID":"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae","Type":"ContainerDied","Data":"713a72f43a32066cc0936db71c0d431afd1ff01fa8b3fcf3f6c7328ef31bcc8c"}
May 25 13:48:10.923263 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:10.923226    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg" event={"ID":"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae","Type":"ContainerStarted","Data":"44082f199606566ea90438fbf7d0482ad131269d8baa481feb36a1a267d8c885"}
May 25 13:48:10.923263 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:10.923266    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg" event={"ID":"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae","Type":"ContainerStarted","Data":"ee8cb8a8075c653c10d85cad422deef0b699194c3820620353234224d9fafb19"}
May 25 13:48:10.923752 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:10.923413    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:10.945513 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:10.945462    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg" podStartSLOduration=2.945444879 podStartE2EDuration="2.945444879s" podCreationTimestamp="2026-05-25 13:48:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:48:10.944245873 +0000 UTC m=+2557.663209013" watchObservedRunningTime="2026-05-25 13:48:10.945444879 +0000 UTC m=+2557.664408019"
May 25 13:48:18.584885 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:18.584849    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:18.584885 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:18.584894    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:18.587840 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:18.587797    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:18.959766 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:18.959660    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:48:39.964184 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:48:39.964151    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:49:49.366021 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:49.365975    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"]
May 25 13:49:49.366569 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:49.366297    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg" podUID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerName="main" containerID="cri-o://ee8cb8a8075c653c10d85cad422deef0b699194c3820620353234224d9fafb19" gracePeriod=30
May 25 13:49:49.366569 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:49.366334    2559 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg" podUID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerName="tokenizer" containerID="cri-o://44082f199606566ea90438fbf7d0482ad131269d8baa481feb36a1a267d8c885" gracePeriod=30
May 25 13:49:49.963407 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:49:49.963372    2559 logging.go:55] [core] [Channel #730 SubChannel #731]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.54:9003", ServerName: "10.133.0.54:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.54:9003: connect: connection refused"
May 25 13:49:50.296195 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.296165    2559 generic.go:358] "Generic (PLEG): container finished" podID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerID="ee8cb8a8075c653c10d85cad422deef0b699194c3820620353234224d9fafb19" exitCode=0
May 25 13:49:50.296371 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.296204    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg" event={"ID":"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae","Type":"ContainerDied","Data":"ee8cb8a8075c653c10d85cad422deef0b699194c3820620353234224d9fafb19"}
May 25 13:49:50.621564 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.621539    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:49:50.780002 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.779967    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tls-certs\") pod \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") "
May 25 13:49:50.780197 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.780008    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-uds\") pod \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") "
May 25 13:49:50.780197 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.780096    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-cache\") pod \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") "
May 25 13:49:50.780197 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.780132    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-kserve-provision-location\") pod \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") "
May 25 13:49:50.780197 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.780189    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g78fl\" (UniqueName: \"kubernetes.io/projected/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-kube-api-access-g78fl\") pod \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") "
May 25 13:49:50.780397 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.780212    2559 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-tmp\") pod \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\" (UID: \"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae\") "
May 25 13:49:50.780397 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.780328    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-uds" (OuterVolumeSpecName: "tokenizer-uds") pod "ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" (UID: "ebc2ca11-e6eb-4eda-94e3-971c863ce1ae"). InnerVolumeSpecName "tokenizer-uds". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:49:50.780397 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.780364    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-cache" (OuterVolumeSpecName: "tokenizer-cache") pod "ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" (UID: "ebc2ca11-e6eb-4eda-94e3-971c863ce1ae"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:49:50.780535 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.780517    2559 reconciler_common.go:299] "Volume detached for volume \"tokenizer-uds\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-uds\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:49:50.780584 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.780543    2559 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-cache\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:49:50.780630 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.780609    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-tmp" (OuterVolumeSpecName: "tokenizer-tmp") pod "ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" (UID: "ebc2ca11-e6eb-4eda-94e3-971c863ce1ae"). InnerVolumeSpecName "tokenizer-tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:49:50.780872 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.780853    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" (UID: "ebc2ca11-e6eb-4eda-94e3-971c863ce1ae"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 13:49:50.782087 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.782067    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" (UID: "ebc2ca11-e6eb-4eda-94e3-971c863ce1ae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 13:49:50.782229 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.782208    2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-kube-api-access-g78fl" (OuterVolumeSpecName: "kube-api-access-g78fl") pod "ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" (UID: "ebc2ca11-e6eb-4eda-94e3-971c863ce1ae"). InnerVolumeSpecName "kube-api-access-g78fl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 13:49:50.881311 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.881228    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g78fl\" (UniqueName: \"kubernetes.io/projected/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-kube-api-access-g78fl\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:49:50.881311 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.881256    2559 reconciler_common.go:299] "Volume detached for volume \"tokenizer-tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tokenizer-tmp\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:49:50.881311 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.881267    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-tls-certs\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:49:50.881311 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.881277    2559 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae-kserve-provision-location\") on node \"ip-10-0-129-151.ec2.internal\" DevicePath \"\""
May 25 13:49:50.963943 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:50.963903    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg" podUID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerName="main" probeResult="failure" output="timeout: failed to connect service \"10.133.0.54:9003\" within 1s: context deadline exceeded"
May 25 13:49:50.964095 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:49:50.964057    2559 logging.go:55] [core] [Channel #730 SubChannel #731]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.54:9003", ServerName: "10.133.0.54:9003", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.54:9003: operation was canceled"
May 25 13:49:51.301775 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.301742    2559 generic.go:358] "Generic (PLEG): container finished" podID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerID="44082f199606566ea90438fbf7d0482ad131269d8baa481feb36a1a267d8c885" exitCode=0
May 25 13:49:51.302008 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.301818    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg" event={"ID":"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae","Type":"ContainerDied","Data":"44082f199606566ea90438fbf7d0482ad131269d8baa481feb36a1a267d8c885"}
May 25 13:49:51.302008 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.301832    2559 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"
May 25 13:49:51.302008 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.301859    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg" event={"ID":"ebc2ca11-e6eb-4eda-94e3-971c863ce1ae","Type":"ContainerDied","Data":"c5acf1dd1532c4881f8d0e0b304a654f9f089f0e7bda6720916b14680c8f28cd"}
May 25 13:49:51.302008 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.301894    2559 scope.go:117] "RemoveContainer" containerID="44082f199606566ea90438fbf7d0482ad131269d8baa481feb36a1a267d8c885"
May 25 13:49:51.311208 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.311189    2559 scope.go:117] "RemoveContainer" containerID="ee8cb8a8075c653c10d85cad422deef0b699194c3820620353234224d9fafb19"
May 25 13:49:51.320680 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.320647    2559 scope.go:117] "RemoveContainer" containerID="713a72f43a32066cc0936db71c0d431afd1ff01fa8b3fcf3f6c7328ef31bcc8c"
May 25 13:49:51.326120 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.326092    2559 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"]
May 25 13:49:51.331374 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.331331    2559 scope.go:117] "RemoveContainer" containerID="44082f199606566ea90438fbf7d0482ad131269d8baa481feb36a1a267d8c885"
May 25 13:49:51.335878 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:49:51.331709    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44082f199606566ea90438fbf7d0482ad131269d8baa481feb36a1a267d8c885\": container with ID starting with 44082f199606566ea90438fbf7d0482ad131269d8baa481feb36a1a267d8c885 not found: ID does not exist" containerID="44082f199606566ea90438fbf7d0482ad131269d8baa481feb36a1a267d8c885"
May 25 13:49:51.335878 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.331762    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44082f199606566ea90438fbf7d0482ad131269d8baa481feb36a1a267d8c885"} err="failed to get container status \"44082f199606566ea90438fbf7d0482ad131269d8baa481feb36a1a267d8c885\": rpc error: code = NotFound desc = could not find container \"44082f199606566ea90438fbf7d0482ad131269d8baa481feb36a1a267d8c885\": container with ID starting with 44082f199606566ea90438fbf7d0482ad131269d8baa481feb36a1a267d8c885 not found: ID does not exist"
May 25 13:49:51.335878 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.331804    2559 scope.go:117] "RemoveContainer" containerID="ee8cb8a8075c653c10d85cad422deef0b699194c3820620353234224d9fafb19"
May 25 13:49:51.335878 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:49:51.332027    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee8cb8a8075c653c10d85cad422deef0b699194c3820620353234224d9fafb19\": container with ID starting with ee8cb8a8075c653c10d85cad422deef0b699194c3820620353234224d9fafb19 not found: ID does not exist" containerID="ee8cb8a8075c653c10d85cad422deef0b699194c3820620353234224d9fafb19"
May 25 13:49:51.335878 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.332058    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee8cb8a8075c653c10d85cad422deef0b699194c3820620353234224d9fafb19"} err="failed to get container status \"ee8cb8a8075c653c10d85cad422deef0b699194c3820620353234224d9fafb19\": rpc error: code = NotFound desc = could not find container \"ee8cb8a8075c653c10d85cad422deef0b699194c3820620353234224d9fafb19\": container with ID starting with ee8cb8a8075c653c10d85cad422deef0b699194c3820620353234224d9fafb19 not found: ID does not exist"
May 25 13:49:51.335878 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.332086    2559 scope.go:117] "RemoveContainer" containerID="713a72f43a32066cc0936db71c0d431afd1ff01fa8b3fcf3f6c7328ef31bcc8c"
May 25 13:49:51.335878 ip-10-0-129-151 kubenswrapper[2559]: E0525 13:49:51.332302    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"713a72f43a32066cc0936db71c0d431afd1ff01fa8b3fcf3f6c7328ef31bcc8c\": container with ID starting with 713a72f43a32066cc0936db71c0d431afd1ff01fa8b3fcf3f6c7328ef31bcc8c not found: ID does not exist" containerID="713a72f43a32066cc0936db71c0d431afd1ff01fa8b3fcf3f6c7328ef31bcc8c"
May 25 13:49:51.335878 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.332351    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"713a72f43a32066cc0936db71c0d431afd1ff01fa8b3fcf3f6c7328ef31bcc8c"} err="failed to get container status \"713a72f43a32066cc0936db71c0d431afd1ff01fa8b3fcf3f6c7328ef31bcc8c\": rpc error: code = NotFound desc = could not find container \"713a72f43a32066cc0936db71c0d431afd1ff01fa8b3fcf3f6c7328ef31bcc8c\": container with ID starting with 713a72f43a32066cc0936db71c0d431afd1ff01fa8b3fcf3f6c7328ef31bcc8c not found: ID does not exist"
May 25 13:49:51.335878 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.333625    2559 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-5c8954d4b9-jrrwg"]
May 25 13:49:51.870801 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:51.870760    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" path="/var/lib/kubelet/pods/ebc2ca11-e6eb-4eda-94e3-971c863ce1ae/volumes"
May 25 13:49:52.137789 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.137680    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz"]
May 25 13:49:52.138066 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.138053    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerName="tokenizer"
May 25 13:49:52.138119 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.138068    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerName="tokenizer"
May 25 13:49:52.138119 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.138102    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerName="main"
May 25 13:49:52.138119 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.138107    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerName="main"
May 25 13:49:52.138219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.138123    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerName="storage-initializer"
May 25 13:49:52.138219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.138129    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerName="storage-initializer"
May 25 13:49:52.138219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.138177    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerName="tokenizer"
May 25 13:49:52.138219 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.138189    2559 memory_manager.go:356] "RemoveStaleState removing state" podUID="ebc2ca11-e6eb-4eda-94e3-971c863ce1ae" containerName="main"
May 25 13:49:52.142649 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.142631    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz"
May 25 13:49:52.145064 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.145038    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-r9zrq\""
May 25 13:49:52.145690 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.145671    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
May 25 13:49:52.149878 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.149773    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz"]
May 25 13:49:52.294492 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.294460    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0c1b65e-9f2e-4af8-9c6e-fead10435b01-cert\") pod \"llmisvc-controller-manager-7f8fbc755f-ts9mz\" (UID: \"f0c1b65e-9f2e-4af8-9c6e-fead10435b01\") " pod="kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz"
May 25 13:49:52.294639 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.294502    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx6hb\" (UniqueName: \"kubernetes.io/projected/f0c1b65e-9f2e-4af8-9c6e-fead10435b01-kube-api-access-lx6hb\") pod \"llmisvc-controller-manager-7f8fbc755f-ts9mz\" (UID: \"f0c1b65e-9f2e-4af8-9c6e-fead10435b01\") " pod="kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz"
May 25 13:49:52.396085 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.395999    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lx6hb\" (UniqueName: \"kubernetes.io/projected/f0c1b65e-9f2e-4af8-9c6e-fead10435b01-kube-api-access-lx6hb\") pod \"llmisvc-controller-manager-7f8fbc755f-ts9mz\" (UID: \"f0c1b65e-9f2e-4af8-9c6e-fead10435b01\") " pod="kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz"
May 25 13:49:52.396239 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.396104    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0c1b65e-9f2e-4af8-9c6e-fead10435b01-cert\") pod \"llmisvc-controller-manager-7f8fbc755f-ts9mz\" (UID: \"f0c1b65e-9f2e-4af8-9c6e-fead10435b01\") " pod="kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz"
May 25 13:49:52.398375 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.398345    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0c1b65e-9f2e-4af8-9c6e-fead10435b01-cert\") pod \"llmisvc-controller-manager-7f8fbc755f-ts9mz\" (UID: \"f0c1b65e-9f2e-4af8-9c6e-fead10435b01\") " pod="kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz"
May 25 13:49:52.405502 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.405480    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx6hb\" (UniqueName: \"kubernetes.io/projected/f0c1b65e-9f2e-4af8-9c6e-fead10435b01-kube-api-access-lx6hb\") pod \"llmisvc-controller-manager-7f8fbc755f-ts9mz\" (UID: \"f0c1b65e-9f2e-4af8-9c6e-fead10435b01\") " pod="kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz"
May 25 13:49:52.453422 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.453381    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz"
May 25 13:49:52.582376 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:52.582347    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz"]
May 25 13:49:52.584191 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:49:52.584155    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf0c1b65e_9f2e_4af8_9c6e_fead10435b01.slice/crio-8ab57e4acff29d7eaf6065f678ab06b244feb10da896e57972cce79d0b0982ad WatchSource:0}: Error finding container 8ab57e4acff29d7eaf6065f678ab06b244feb10da896e57972cce79d0b0982ad: Status 404 returned error can't find the container with id 8ab57e4acff29d7eaf6065f678ab06b244feb10da896e57972cce79d0b0982ad
May 25 13:49:53.311917 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:53.311885    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz" event={"ID":"f0c1b65e-9f2e-4af8-9c6e-fead10435b01","Type":"ContainerStarted","Data":"8ab57e4acff29d7eaf6065f678ab06b244feb10da896e57972cce79d0b0982ad"}
May 25 13:49:55.320440 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:55.320409    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz" event={"ID":"f0c1b65e-9f2e-4af8-9c6e-fead10435b01","Type":"ContainerStarted","Data":"bbe101c38572b76ec2a4d4cff28ff3df96062f5b4fdccac3d2c419acdba00b4a"}
May 25 13:49:55.320820 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:55.320514    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz"
May 25 13:49:55.340451 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:49:55.340399    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz" podStartSLOduration=0.81218821 podStartE2EDuration="3.340385559s" podCreationTimestamp="2026-05-25 13:49:52 +0000 UTC" firstStartedPulling="2026-05-25 13:49:52.585492677 +0000 UTC m=+2659.304455793" lastFinishedPulling="2026-05-25 13:49:55.113690019 +0000 UTC m=+2661.832653142" observedRunningTime="2026-05-25 13:49:55.337488092 +0000 UTC m=+2662.056451233" watchObservedRunningTime="2026-05-25 13:49:55.340385559 +0000 UTC m=+2662.059348696"
May 25 13:50:26.326514 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:50:26.326432    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-7f8fbc755f-ts9mz"
May 25 13:58:46.872965 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:46.872932    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"]
May 25 13:58:46.876352 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:46.876330    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:46.878591 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:46.878567    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"router-gateway-2-openshift-default-dockercfg-lphth\""
May 25 13:58:46.897028 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:46.897000    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"]
May 25 13:58:46.987457 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:46.987418    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-istio-data\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:46.987650 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:46.987469    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-credential-socket\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:46.987650 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:46.987517    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/217242c4-5b8c-4527-a5ff-b00a2d54f079-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:46.987650 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:46.987546    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-workload-socket\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:46.987650 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:46.987620    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/217242c4-5b8c-4527-a5ff-b00a2d54f079-istio-token\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:46.987908 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:46.987698    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/217242c4-5b8c-4527-a5ff-b00a2d54f079-istio-podinfo\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:46.987908 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:46.987789    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-istio-envoy\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:46.987908 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:46.987823    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-workload-certs\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:46.987908 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:46.987882    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w47x\" (UniqueName: \"kubernetes.io/projected/217242c4-5b8c-4527-a5ff-b00a2d54f079-kube-api-access-6w47x\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.088896 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.088857    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/217242c4-5b8c-4527-a5ff-b00a2d54f079-istio-podinfo\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.089092 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.088909    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-istio-envoy\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.089092 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.088942    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-workload-certs\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.089092 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.088970    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6w47x\" (UniqueName: \"kubernetes.io/projected/217242c4-5b8c-4527-a5ff-b00a2d54f079-kube-api-access-6w47x\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.089092 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.089003    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-istio-data\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.089092 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.089045    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-credential-socket\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.089092 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.089077    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/217242c4-5b8c-4527-a5ff-b00a2d54f079-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.089400 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.089113    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-workload-socket\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.089453 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.089419    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-workload-certs\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.089453 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.089444    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-istio-data\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.089576 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.089490    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-credential-socket\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.089700 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.089673    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-workload-socket\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.089903 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.089746    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/217242c4-5b8c-4527-a5ff-b00a2d54f079-istio-token\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.089977 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.089891    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/217242c4-5b8c-4527-a5ff-b00a2d54f079-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.091391 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.091369    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/217242c4-5b8c-4527-a5ff-b00a2d54f079-istio-envoy\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.091590 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.091569    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/217242c4-5b8c-4527-a5ff-b00a2d54f079-istio-podinfo\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.098229 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.098204    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w47x\" (UniqueName: \"kubernetes.io/projected/217242c4-5b8c-4527-a5ff-b00a2d54f079-kube-api-access-6w47x\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.098501 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.098479    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/217242c4-5b8c-4527-a5ff-b00a2d54f079-istio-token\") pod \"router-gateway-2-openshift-default-7bb57db77b-4clmr\" (UID: \"217242c4-5b8c-4527-a5ff-b00a2d54f079\") " pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.188322 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.188232    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:47.338541 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.338395    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"]
May 25 13:58:47.341140 ip-10-0-129-151 kubenswrapper[2559]: W0525 13:58:47.341110    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod217242c4_5b8c_4527_a5ff_b00a2d54f079.slice/crio-f1a1dc411a1103fb0ccb6ba8c7da197d69090eff8a7226d0e6d0058cfb677d1d WatchSource:0}: Error finding container f1a1dc411a1103fb0ccb6ba8c7da197d69090eff8a7226d0e6d0058cfb677d1d: Status 404 returned error can't find the container with id f1a1dc411a1103fb0ccb6ba8c7da197d69090eff8a7226d0e6d0058cfb677d1d
May 25 13:58:47.343054 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.343035    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 13:58:47.343448 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.343415    2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 13:58:47.343536 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.343496    2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 13:58:47.343536 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:47.343532    2559 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
May 25 13:58:48.337016 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:48.336980    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr" event={"ID":"217242c4-5b8c-4527-a5ff-b00a2d54f079","Type":"ContainerStarted","Data":"b15ec95e11b5cd3304c87777cd3b0a953bc37e8bccc010cd2b208e30bab75b46"}
May 25 13:58:48.337391 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:48.337023    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr" event={"ID":"217242c4-5b8c-4527-a5ff-b00a2d54f079","Type":"ContainerStarted","Data":"f1a1dc411a1103fb0ccb6ba8c7da197d69090eff8a7226d0e6d0058cfb677d1d"}
May 25 13:58:48.359794 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:48.359736    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr" podStartSLOduration=2.359701335 podStartE2EDuration="2.359701335s" podCreationTimestamp="2026-05-25 13:58:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 13:58:48.357163839 +0000 UTC m=+3195.076126978" watchObservedRunningTime="2026-05-25 13:58:48.359701335 +0000 UTC m=+3195.078664479"
May 25 13:58:49.189300 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:49.189263    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:50.202010 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:50.201956    2559 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr" podUID="217242c4-5b8c-4527-a5ff-b00a2d54f079" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
May 25 13:58:51.202932 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:51.202877    2559 prober.go:120] "Probe failed" probeType="Startup" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr" podUID="217242c4-5b8c-4527-a5ff-b00a2d54f079" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
May 25 13:58:51.462778 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:51.462651    2559 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:51.463158 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:51.463138    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 13:58:51.464277 ip-10-0-129-151 kubenswrapper[2559]: I0525 13:58:51.464254    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/router-gateway-2-openshift-default-7bb57db77b-4clmr"
May 25 14:09:55.452275 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:09:55.452204    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:09:55.482563 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:09:55.482532    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:09:56.366734 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:09:56.366701    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:09:56.382574 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:09:56.382547    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:09:57.283919 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:09:57.283888    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:09:57.298615 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:09:57.298593    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:09:58.171058 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:09:58.171023    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:09:58.185813 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:09:58.185784    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:09:59.079779 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:09:59.079748    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:09:59.096313 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:09:59.096279    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:09:59.974737 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:09:59.974684    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:09:59.999836 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:09:59.999811    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:10:00.902921 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:00.902892    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:10:00.921673 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:00.921649    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:10:01.830475 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:01.830446    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:10:01.846094 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:01.846069    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:10:02.743073 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:02.743040    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:10:02.758028 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:02.757996    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:10:03.654410 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:03.654383    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:10:03.669626 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:03.669597    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:10:04.556872 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:04.556846    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:10:04.571896 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:04.571872    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:10:05.448664 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:05.448631    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:10:05.463582 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:05.463559    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:10:06.397867 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:06.397840    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:10:06.415121 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:06.415087    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:10:07.360905 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:07.360876    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-1-openshift-default-7b9cf87876-w7tqm_464b6cc1-f77b-4c8d-98e1-2c9fedfaa765/istio-proxy/0.log"
May 25 14:10:07.388061 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:07.388024    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_router-gateway-2-openshift-default-7bb57db77b-4clmr_217242c4-5b8c-4527-a5ff-b00a2d54f079/istio-proxy/0.log"
May 25 14:10:08.365048 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:08.365019    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-6p94s_598289c4-26ea-49fc-8e24-c662d5be93d8/discovery/0.log"
May 25 14:10:08.480583 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:08.480550    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw_4f70df2d-eb18-4ee6-b889-03a6fbbd5081/istio-proxy/0.log"
May 25 14:10:09.291847 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:09.291818    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-6p94s_598289c4-26ea-49fc-8e24-c662d5be93d8/discovery/0.log"
May 25 14:10:09.392376 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:09.392346    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw_4f70df2d-eb18-4ee6-b889-03a6fbbd5081/istio-proxy/0.log"
May 25 14:10:10.222559 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:10.222527    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-bqc7m_4f3e4f48-cfe0-405e-8993-de677e493d70/manager/0.log"
May 25 14:10:10.241241 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:10.241216    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-844548ff4c-k8p9k_ee52a3c2-7253-4cb0-96cb-1489027a5c61/manager/0.log"
May 25 14:10:10.296432 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:10.296405    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57dd85bb4-rrgxc_4ce8a068-b18e-41f8-8175-70487ebafe61/manager/0.log"
May 25 14:10:15.416885 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:15.416851    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xfbms_fbb8062f-fbfd-4af1-ae8d-c7fb90612985/global-pull-secret-syncer/0.log"
May 25 14:10:15.542434 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:15.542402    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zjh8s_03bb8722-1d72-4446-8a56-7d37772243a4/konnectivity-agent/0.log"
May 25 14:10:15.565956 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:15.565929    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-151.ec2.internal_709c800460854ebcab9b23cb49bcdc37/haproxy/0.log"
May 25 14:10:19.552289 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:19.552256    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7587b89b76-bqc7m_4f3e4f48-cfe0-405e-8993-de677e493d70/manager/0.log"
May 25 14:10:19.579473 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:19.579444    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-844548ff4c-k8p9k_ee52a3c2-7253-4cb0-96cb-1489027a5c61/manager/0.log"
May 25 14:10:19.665898 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:19.665866    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57dd85bb4-rrgxc_4ce8a068-b18e-41f8-8175-70487ebafe61/manager/0.log"
May 25 14:10:21.164771 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:21.164740    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5wpqv_afba796a-339d-41ed-aaba-9f1aeece3b05/node-exporter/0.log"
May 25 14:10:21.188542 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:21.188501    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5wpqv_afba796a-339d-41ed-aaba-9f1aeece3b05/kube-rbac-proxy/0.log"
May 25 14:10:21.209901 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:21.209876    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5wpqv_afba796a-339d-41ed-aaba-9f1aeece3b05/init-textfile/0.log"
May 25 14:10:23.370774 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:23.370742    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-kb7wb_97b70200-e3c7-46a5-b830-91346bf7cb88/networking-console-plugin/0.log"
May 25 14:10:24.389549 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.389517    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5cc6bf9c98-s9zcq_86e8b3d0-df94-4f49-acf7-702079789dcb/console/0.log"
May 25 14:10:24.426562 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.426528    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-rfml4_a80cd8c7-074e-47ef-8de3-f2921430ad60/download-server/0.log"
May 25 14:10:24.576674 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.576638    2559 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"]
May 25 14:10:24.580324 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.580299    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.582610 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.582589    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tl2jt\"/\"openshift-service-ca.crt\""
May 25 14:10:24.583290 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.583264    2559 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tl2jt\"/\"kube-root-ca.crt\""
May 25 14:10:24.583389 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.583266    2559 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tl2jt\"/\"default-dockercfg-kltt4\""
May 25 14:10:24.591365 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.591338    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"]
May 25 14:10:24.746258 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.746229    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06309c25-209c-4eaa-93ce-f21c3177aa86-sys\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.746415 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.746276    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sstp\" (UniqueName: \"kubernetes.io/projected/06309c25-209c-4eaa-93ce-f21c3177aa86-kube-api-access-7sstp\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.746415 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.746313    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/06309c25-209c-4eaa-93ce-f21c3177aa86-proc\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.746415 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.746336    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/06309c25-209c-4eaa-93ce-f21c3177aa86-podres\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.746415 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.746390    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/06309c25-209c-4eaa-93ce-f21c3177aa86-lib-modules\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.847080 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.847043    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/06309c25-209c-4eaa-93ce-f21c3177aa86-lib-modules\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.847249 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.847140    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06309c25-209c-4eaa-93ce-f21c3177aa86-sys\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.847249 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.847181    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7sstp\" (UniqueName: \"kubernetes.io/projected/06309c25-209c-4eaa-93ce-f21c3177aa86-kube-api-access-7sstp\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.847249 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.847219    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/06309c25-209c-4eaa-93ce-f21c3177aa86-proc\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.847249 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.847233    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/06309c25-209c-4eaa-93ce-f21c3177aa86-lib-modules\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.847249 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.847242    2559 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/06309c25-209c-4eaa-93ce-f21c3177aa86-podres\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.847501 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.847258    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06309c25-209c-4eaa-93ce-f21c3177aa86-sys\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.847501 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.847293    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/06309c25-209c-4eaa-93ce-f21c3177aa86-proc\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.847501 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.847395    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/06309c25-209c-4eaa-93ce-f21c3177aa86-podres\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.854965 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.854938    2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sstp\" (UniqueName: \"kubernetes.io/projected/06309c25-209c-4eaa-93ce-f21c3177aa86-kube-api-access-7sstp\") pod \"perf-node-gather-daemonset-8cvh2\" (UID: \"06309c25-209c-4eaa-93ce-f21c3177aa86\") " pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:24.893219 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:24.893190    2559 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:25.022620 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:25.022448    2559 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"]
May 25 14:10:25.024881 ip-10-0-129-151 kubenswrapper[2559]: W0525 14:10:25.024847    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod06309c25_209c_4eaa_93ce_f21c3177aa86.slice/crio-903d0c079c48987607a9c71b7fdd53488b0972c8f2ebd4a58db1fc6fd590456d WatchSource:0}: Error finding container 903d0c079c48987607a9c71b7fdd53488b0972c8f2ebd4a58db1fc6fd590456d: Status 404 returned error can't find the container with id 903d0c079c48987607a9c71b7fdd53488b0972c8f2ebd4a58db1fc6fd590456d
May 25 14:10:25.026469 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:25.026452    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 14:10:25.695183 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:25.695146    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f6lwr_7844394a-a353-4e6f-9c29-75e111adcdb9/dns/0.log"
May 25 14:10:25.719844 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:25.719817    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f6lwr_7844394a-a353-4e6f-9c29-75e111adcdb9/kube-rbac-proxy/0.log"
May 25 14:10:25.800960 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:25.800932    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-482g6_9ea8b5d3-ab26-4393-b5f9-747d95405510/dns-node-resolver/0.log"
May 25 14:10:25.989014 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:25.988923    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2" event={"ID":"06309c25-209c-4eaa-93ce-f21c3177aa86","Type":"ContainerStarted","Data":"b13cbb32466cfdd01b3ec9d5a56cc5ee44f71821f8a6b18eba765b899b8c99ed"}
May 25 14:10:25.989014 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:25.988968    2559 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2" event={"ID":"06309c25-209c-4eaa-93ce-f21c3177aa86","Type":"ContainerStarted","Data":"903d0c079c48987607a9c71b7fdd53488b0972c8f2ebd4a58db1fc6fd590456d"}
May 25 14:10:25.989193 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:25.989053    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:26.005761 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:26.005700    2559 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2" podStartSLOduration=2.005684117 podStartE2EDuration="2.005684117s" podCreationTimestamp="2026-05-25 14:10:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 14:10:26.004414824 +0000 UTC m=+3892.723377965" watchObservedRunningTime="2026-05-25 14:10:26.005684117 +0000 UTC m=+3892.724647256"
May 25 14:10:26.349439 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:26.349408    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-bbd6764c5-zll4g_b1f9b575-90ca-41e5-b2c0-c90137e06eac/registry/0.log"
May 25 14:10:26.374727 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:26.374689    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-95pwd_bc716b46-7c31-42f5-ba6f-45b8c77d29ad/node-ca/0.log"
May 25 14:10:27.322635 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:27.322606    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-6p94s_598289c4-26ea-49fc-8e24-c662d5be93d8/discovery/0.log"
May 25 14:10:27.349519 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:27.349496    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-849cdfbf5f-6jlgw_4f70df2d-eb18-4ee6-b889-03a6fbbd5081/istio-proxy/0.log"
May 25 14:10:27.892453 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:27.892423    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lftth_95c09ce4-ed3a-4de9-baef-5b4e1fed5652/serve-healthcheck-canary/0.log"
May 25 14:10:28.551081 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:28.551055    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n7bht_989dfcac-2684-4ca2-929d-947ffe34fba1/kube-rbac-proxy/0.log"
May 25 14:10:28.575574 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:28.575546    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n7bht_989dfcac-2684-4ca2-929d-947ffe34fba1/exporter/0.log"
May 25 14:10:28.601319 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:28.601289    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n7bht_989dfcac-2684-4ca2-929d-947ffe34fba1/extractor/0.log"
May 25 14:10:31.848485 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:31.848455    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-7f8fbc755f-ts9mz_f0c1b65e-9f2e-4af8-9c6e-fead10435b01/manager/0.log"
May 25 14:10:32.003427 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:32.003399    2559 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tl2jt/perf-node-gather-daemonset-8cvh2"
May 25 14:10:32.084003 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:32.083969    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-696fc77849-9ntd8_fd43edef-f491-4b47-ab13-002d03a58f00/manager/0.log"
May 25 14:10:32.108173 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:32.108089    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-82g57_4304656e-7e95-4287-96d4-320989cd5888/s3-init/0.log"
May 25 14:10:32.138617 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:32.138589    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-79f457656b-s4xkh_d74918be-6d2c-4d4d-8d8f-6c05b2c2b7b0/seaweedfs/0.log"
May 25 14:10:36.916173 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:36.916138    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-zcpd6_9ff1dfe7-ee99-4531-a9fc-ace686d55b62/migrator/0.log"
May 25 14:10:36.948784 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:36.948753    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-zcpd6_9ff1dfe7-ee99-4531-a9fc-ace686d55b62/graceful-termination/0.log"
May 25 14:10:38.398195 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:38.398166    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lvfw_913e5f01-6ff3-4bd5-965e-809823df7d6d/kube-multus-additional-cni-plugins/0.log"
May 25 14:10:38.429520 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:38.429493    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lvfw_913e5f01-6ff3-4bd5-965e-809823df7d6d/egress-router-binary-copy/0.log"
May 25 14:10:38.449173 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:38.449153    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lvfw_913e5f01-6ff3-4bd5-965e-809823df7d6d/cni-plugins/0.log"
May 25 14:10:38.470070 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:38.470047    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lvfw_913e5f01-6ff3-4bd5-965e-809823df7d6d/bond-cni-plugin/0.log"
May 25 14:10:38.490571 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:38.490548    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lvfw_913e5f01-6ff3-4bd5-965e-809823df7d6d/routeoverride-cni/0.log"
May 25 14:10:38.511691 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:38.511659    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lvfw_913e5f01-6ff3-4bd5-965e-809823df7d6d/whereabouts-cni-bincopy/0.log"
May 25 14:10:38.539072 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:38.539048    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9lvfw_913e5f01-6ff3-4bd5-965e-809823df7d6d/whereabouts-cni/0.log"
May 25 14:10:39.000074 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:39.000039    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tghhw_c886a9b9-c5da-4569-a695-125c80d706f6/kube-multus/0.log"
May 25 14:10:39.116469 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:39.116440    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tx5hr_fde8fc4f-07bb-442f-aa5e-6e63efece856/network-metrics-daemon/0.log"
May 25 14:10:39.136731 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:39.136700    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tx5hr_fde8fc4f-07bb-442f-aa5e-6e63efece856/kube-rbac-proxy/0.log"
May 25 14:10:40.664827 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:40.664745    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x4vqs_724bf4c7-1d41-4ddf-aa5e-d8b043c08c35/ovn-controller/0.log"
May 25 14:10:40.698326 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:40.698289    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x4vqs_724bf4c7-1d41-4ddf-aa5e-d8b043c08c35/ovn-acl-logging/0.log"
May 25 14:10:40.716661 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:40.716639    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x4vqs_724bf4c7-1d41-4ddf-aa5e-d8b043c08c35/kube-rbac-proxy-node/0.log"
May 25 14:10:40.742253 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:40.742234    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x4vqs_724bf4c7-1d41-4ddf-aa5e-d8b043c08c35/kube-rbac-proxy-ovn-metrics/0.log"
May 25 14:10:40.763809 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:40.763783    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x4vqs_724bf4c7-1d41-4ddf-aa5e-d8b043c08c35/northd/0.log"
May 25 14:10:40.790424 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:40.790398    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x4vqs_724bf4c7-1d41-4ddf-aa5e-d8b043c08c35/nbdb/0.log"
May 25 14:10:40.812673 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:40.812649    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x4vqs_724bf4c7-1d41-4ddf-aa5e-d8b043c08c35/sbdb/0.log"
May 25 14:10:40.916372 ip-10-0-129-151 kubenswrapper[2559]: I0525 14:10:40.916284    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-x4vqs_724bf4c7-1d41-4ddf-aa5e-d8b043c08c35/ovnkube-controller/0.log"