May 28 03:16:47.168836 ip-10-0-141-36 systemd[1]: Starting Kubernetes Kubelet... May 28 03:16:47.630329 ip-10-0-141-36 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 03:16:47.630329 ip-10-0-141-36 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 28 03:16:47.630329 ip-10-0-141-36 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 03:16:47.630329 ip-10-0-141-36 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 28 03:16:47.630329 ip-10-0-141-36 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 03:16:47.632140 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.631948 2555 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 28 03:16:47.636084 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636066 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 03:16:47.636084 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636085 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636089 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636092 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636097 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636100 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636103 2555 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636105 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636108 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636111 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636113 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636116 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636119 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636122 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636124 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636127 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636130 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636141 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636144 2555 feature_gate.go:328] unrecognized feature gate: Example2 May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636146 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636149 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 03:16:47.636154 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636152 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636154 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636157 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636160 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636163 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636165 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636168 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636171 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636173 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636176 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636179 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636181 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636184 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636187 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636190 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636193 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636195 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636198 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636200 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 03:16:47.636649 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636203 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636206 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636208 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636211 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636214 2555 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636217 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636219 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636222 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636224 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636227 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636231 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636234 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636236 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636239 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636241 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636244 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636247 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636250 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636253 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636255 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 03:16:47.637109 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636258 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636261 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636263 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636265 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636268 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636272 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636275 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636278 2555 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636280 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636283 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636285 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636288 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636290 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636293 2555 feature_gate.go:328] unrecognized feature gate: Example May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636297 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636301 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636303 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636306 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636308 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 03:16:47.637596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636312 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 03:16:47.638050 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636316 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 03:16:47.638050 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636319 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 03:16:47.638050 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636322 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 03:16:47.638050 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636325 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 03:16:47.638050 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636328 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 03:16:47.638050 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.636331 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 03:16:47.638341 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638329 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 03:16:47.638341 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638340 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638344 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638347 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638350 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638353 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638356 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638359 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638361 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638364 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638368 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638371 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638374 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638377 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638380 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638383 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638386 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638389 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638393 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638397 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 03:16:47.638395 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638401 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638404 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638407 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638410 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638413 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638415 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638418 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638420 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638423 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638425 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638428 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638430 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638433 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638436 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638439 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638441 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638444 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638447 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638450 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638452 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 03:16:47.638914 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638454 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638457 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638459 2555 feature_gate.go:328] unrecognized feature gate: Example May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638463 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638465 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638468 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638470 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638473 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638475 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638478 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638480 2555 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638482 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638485 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638501 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638505 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638507 2555 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638510 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638513 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638515 2555 feature_gate.go:328] unrecognized feature gate: Example2 May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638518 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 03:16:47.639407 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638521 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638523 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638526 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638528 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638531 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638534 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638536 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638539 2555 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638541 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638544 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638546 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638549 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638552 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638554 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638557 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638559 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638562 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638565 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638568 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638570 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 03:16:47.639916 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638573 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638575 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638578 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638580 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638583 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.638585 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638675 2555 flags.go:64] FLAG: --address="0.0.0.0" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638683 2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638690 2555 flags.go:64] FLAG: --anonymous-auth="true" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638695 2555 flags.go:64] FLAG: --application-metrics-count-limit="100" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638701 2555 flags.go:64] FLAG: --authentication-token-webhook="false" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638704 2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638709 2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638713 2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638717 2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638720 2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638723 2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638727 2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638730 2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638733 2555 flags.go:64] FLAG: --cgroup-root="" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638736 2555 flags.go:64] FLAG: --cgroups-per-qos="true" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638739 2555 flags.go:64] FLAG: --client-ca-file="" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638742 2555 flags.go:64] FLAG: --cloud-config="" May 28 03:16:47.640400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638745 2555 flags.go:64] FLAG: --cloud-provider="external" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638749 2555 flags.go:64] FLAG: --cluster-dns="[]" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638753 2555 flags.go:64] FLAG: --cluster-domain="" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638756 2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638760 2555 flags.go:64] FLAG: --config-dir="" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638763 2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638766 2555 flags.go:64] FLAG: --container-log-max-files="5" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638770 2555 flags.go:64] FLAG: --container-log-max-size="10Mi" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638773 2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638777 2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638780 2555 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638783 2555 flags.go:64] FLAG: --contention-profiling="false" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638787 2555 flags.go:64] FLAG: --cpu-cfs-quota="true" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638790 2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638793 2555 flags.go:64] FLAG: --cpu-manager-policy="none" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638798 2555 flags.go:64] FLAG: --cpu-manager-policy-options="" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638803 2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638806 2555 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638810 2555 flags.go:64] FLAG: --enable-debugging-handlers="true" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638813 2555 flags.go:64] FLAG: --enable-load-reader="false" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638816 2555 flags.go:64] FLAG: --enable-server="true" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638819 2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638824 2555 flags.go:64] FLAG: --event-burst="100" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638827 2555 flags.go:64] FLAG: --event-qps="50" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638830 2555 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 28 03:16:47.640998 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638833 2555 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638836 2555 flags.go:64] FLAG: --eviction-hard="" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638840 2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638843 2555 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638847 2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638850 2555 flags.go:64] FLAG: --eviction-soft="" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638853 2555 flags.go:64] FLAG: --eviction-soft-grace-period="" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638856 2555 flags.go:64] FLAG: --exit-on-lock-contention="false" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638859 2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638862 2555 flags.go:64] FLAG: --experimental-mounter-path="" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638865 2555 flags.go:64] FLAG: --fail-cgroupv1="false" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638868 2555 flags.go:64] FLAG: --fail-swap-on="true" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638870 2555 flags.go:64] FLAG: --feature-gates="" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638879 2555 flags.go:64] FLAG: --file-check-frequency="20s" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638882 2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638886 2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638889 2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638892 2555 flags.go:64] FLAG: --healthz-port="10248" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638895 2555 flags.go:64] FLAG: --help="false" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638898 2555 flags.go:64] FLAG: --hostname-override="ip-10-0-141-36.ec2.internal" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638901 2555 flags.go:64] FLAG: --housekeeping-interval="10s" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638905 2555 flags.go:64] FLAG: --http-check-frequency="20s" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638909 2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638913 2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 28 03:16:47.641612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638917 2555 flags.go:64] FLAG: --image-gc-high-threshold="85" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638921 2555 flags.go:64] FLAG: --image-gc-low-threshold="80" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638924 2555 flags.go:64] FLAG: --image-service-endpoint="" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638927 2555 flags.go:64] FLAG: --kernel-memcg-notification="false" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638930 2555 flags.go:64] FLAG: --kube-api-burst="100" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638933 2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638936 2555 flags.go:64] FLAG: --kube-api-qps="50" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638939 2555 flags.go:64] FLAG: --kube-reserved="" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638942 2555 flags.go:64] FLAG: --kube-reserved-cgroup="" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638945 2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638949 2555 flags.go:64] FLAG: --kubelet-cgroups="" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638952 2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638955 2555 flags.go:64] FLAG: --lock-file="" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638958 2555 flags.go:64] FLAG: --log-cadvisor-usage="false" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638961 2555 flags.go:64] FLAG: --log-flush-frequency="5s" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638964 2555 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638969 2555 flags.go:64] FLAG: --log-json-split-stream="false" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638972 2555 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638976 2555 flags.go:64] FLAG: --log-text-split-stream="false" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638978 2555 flags.go:64] FLAG: --logging-format="text" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638981 2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638984 2555 flags.go:64] FLAG: --make-iptables-util-chains="true" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638987 2555 flags.go:64] FLAG: --manifest-url="" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638990 2555 flags.go:64] FLAG: --manifest-url-header="" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638995 2555 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 28 03:16:47.642234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.638998 2555 flags.go:64] FLAG: --max-open-files="1000000" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639002 2555 flags.go:64] FLAG: --max-pods="110" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639005 2555 flags.go:64] FLAG: --maximum-dead-containers="-1" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639008 2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639011 2555 flags.go:64] FLAG: --memory-manager-policy="None" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639015 2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639018 2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639021 2555 flags.go:64] FLAG: --node-ip="0.0.0.0" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639025 2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639032 2555 flags.go:64] FLAG: --node-status-max-images="50" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639036 2555 flags.go:64] FLAG: --node-status-update-frequency="10s" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639039 2555 flags.go:64] FLAG: --oom-score-adj="-999" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639042 2555 flags.go:64] FLAG: --pod-cidr="" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639045 2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639052 2555 flags.go:64] FLAG: --pod-manifest-path="" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639055 2555 flags.go:64] FLAG: --pod-max-pids="-1" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639058 2555 flags.go:64] FLAG: --pods-per-core="0" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639061 2555 flags.go:64] FLAG: --port="10250" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639064 2555 flags.go:64] FLAG: --protect-kernel-defaults="false" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639067 2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0546054512bb6cd76" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639070 2555 flags.go:64] FLAG: --qos-reserved="" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639073 2555 flags.go:64] FLAG: --read-only-port="10255" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639076 2555 flags.go:64] FLAG: --register-node="true" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639079 2555 flags.go:64] FLAG: --register-schedulable="true" May 28 03:16:47.642850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639082 2555 flags.go:64] FLAG: --register-with-taints="" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639086 2555 flags.go:64] FLAG: --registry-burst="10" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639089 2555 flags.go:64] FLAG: --registry-qps="5" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639092 2555 flags.go:64] FLAG: --reserved-cpus="" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639095 2555 flags.go:64] FLAG: --reserved-memory="" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639099 2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639102 2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639105 2555 flags.go:64] FLAG: --rotate-certificates="false" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639108 2555 flags.go:64] FLAG: --rotate-server-certificates="false" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639111 2555 flags.go:64] FLAG: --runonce="false" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639113 2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639117 2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639120 2555 flags.go:64] FLAG: --seccomp-default="false" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639123 2555 flags.go:64] FLAG: --serialize-image-pulls="true" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639127 2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639130 2555 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639133 2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639137 2555 flags.go:64] FLAG: --storage-driver-password="root" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639140 2555 flags.go:64] FLAG: --storage-driver-secure="false" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639143 2555 flags.go:64] FLAG: --storage-driver-table="stats" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639145 2555 flags.go:64] FLAG: --storage-driver-user="root" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639148 2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639151 2555 flags.go:64] FLAG: --sync-frequency="1m0s" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639154 2555 flags.go:64] FLAG: --system-cgroups="" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639157 2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 28 03:16:47.643468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639162 2555 flags.go:64] FLAG: --system-reserved-cgroup="" May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639165 2555 flags.go:64] FLAG: --tls-cert-file="" May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639168 2555 flags.go:64] FLAG: --tls-cipher-suites="[]" May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639172 2555 flags.go:64] FLAG: --tls-min-version="" May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639175 2555 flags.go:64] FLAG: --tls-private-key-file="" May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639178 2555 flags.go:64] FLAG: --topology-manager-policy="none" May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639181 2555 flags.go:64] FLAG: --topology-manager-policy-options="" May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639184 2555 flags.go:64] FLAG: --topology-manager-scope="container" May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639187 2555 flags.go:64] FLAG: --v="2" May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639191 2555 flags.go:64] FLAG: --version="false" May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639196 2555 flags.go:64] FLAG: --vmodule="" May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639200 2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.639207 2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639304 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639307 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639311 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639314 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639317 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639319 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639322 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639324 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639327 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639329 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 03:16:47.644094 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639332 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639335 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639337 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639340 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639343 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639345 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639348 2555 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639350 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639353 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639355 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639358 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639360 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639363 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639366 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639368 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639371 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639373 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639375 2555 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639378 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639381 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 03:16:47.644653 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639384 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639388 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639390 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639393 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639396 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639399 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639401 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639404 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639406 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639409 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639413 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639415 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639418 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639421 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639425 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639429 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639432 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639435 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639437 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 03:16:47.645150 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639440 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639442 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639445 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639447 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639452 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639455 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639458 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639460 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639463 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639465 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639469 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639473 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639476 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639479 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639483 2555 feature_gate.go:328] unrecognized feature gate: Example May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639486 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639501 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639503 2555 feature_gate.go:328] unrecognized feature gate: Example2 May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639506 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 03:16:47.645637 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639508 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639511 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639514 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639516 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639519 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639521 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639524 2555 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639526 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639529 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639531 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639534 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639537 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639539 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639542 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639544 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639547 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639549 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 03:16:47.646130 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.639554 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 03:16:47.646568 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.640546 2555 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 03:16:47.647784 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.647761 2555 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 28 03:16:47.647825 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.647786 2555 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 28 03:16:47.647856 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647839 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 03:16:47.647856 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647845 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 03:16:47.647856 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647849 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 03:16:47.647856 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647852 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 03:16:47.647856 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647855 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 03:16:47.647856 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647858 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647861 2555 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647864 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647867 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647869 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647872 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647875 2555 feature_gate.go:328] unrecognized feature gate: Example2 May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647878 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647880 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647883 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647886 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647889 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647891 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647894 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647896 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647899 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647901 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647904 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647907 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647909 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 03:16:47.648032 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647912 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647914 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647917 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647921 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647927 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647931 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647936 2555 feature_gate.go:328] unrecognized feature gate: Example May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647939 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647942 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647944 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647948 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647950 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647953 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647956 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647959 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647962 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647965 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647968 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647971 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 03:16:47.648590 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647973 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647976 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647979 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647982 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647984 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647987 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647990 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647993 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647995 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.647998 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648001 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648003 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648006 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648009 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648020 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648023 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648025 2555 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648028 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648030 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648033 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 03:16:47.649057 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648039 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648042 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648044 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648047 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648050 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648052 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648055 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648059 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648061 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648064 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648067 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648070 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648073 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648076 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648079 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648082 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648084 2555 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648087 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648090 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 03:16:47.649599 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648092 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648095 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648097 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.648102 2555 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648217 2555 feature_gate.go:328] unrecognized feature gate: Example2 May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648223 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648227 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648230 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648232 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648235 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648238 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648241 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648243 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648246 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648256 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 03:16:47.650069 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648259 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648264 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648268 2555 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648271 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648274 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648277 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648280 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648283 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648286 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648289 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648292 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648295 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648298 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648301 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648303 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648306 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648309 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648312 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648314 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 03:16:47.650447 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648317 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648319 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648322 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648324 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648327 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648329 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648332 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648335 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648337 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648339 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648342 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648344 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648347 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648350 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648353 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648355 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648358 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648361 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648364 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648367 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 03:16:47.650925 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648370 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648373 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648375 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648378 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648381 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648383 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648385 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648388 2555 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648390 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648393 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648396 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648398 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648401 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648403 2555 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648406 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648408 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648410 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648414 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648417 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 03:16:47.651411 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648420 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648422 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648425 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648427 2555 feature_gate.go:328] unrecognized feature gate: Example May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648430 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648432 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648435 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648438 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648440 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648443 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648446 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648449 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648452 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648454 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648457 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648459 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 03:16:47.651957 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:47.648462 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 03:16:47.652366 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.648467 2555 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 03:16:47.652366 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.649198 2555 server.go:962] "Client rotation is on, will bootstrap in background" May 28 03:16:47.652366 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.651371 2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 28 03:16:47.652531 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.652518 2555 server.go:1019] "Starting client certificate rotation" May 28 03:16:47.652647 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.652629 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 28 03:16:47.652700 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.652677 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 28 03:16:47.678249 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.678218 2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 28 03:16:47.680368 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.680340 2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 28 03:16:47.695816 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.695787 2555 log.go:25] "Validated CRI v1 runtime API" May 28 03:16:47.700948 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.700916 2555 log.go:25] "Validated CRI v1 image API" May 28 03:16:47.703522 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.703483 2555 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 28 03:16:47.706232 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.706210 2555 fs.go:135] Filesystem UUIDs: map[44b6641e-6ec7-4470-a316-25abcb2704da:/dev/nvme0n1p3 673d0351-ba1a-4a9b-b95a-2735f101eb4f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] May 28 03:16:47.706313 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.706231 2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 28 03:16:47.711420 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.711304 2555 manager.go:217] Machine: {Timestamp:2026-05-28 03:16:47.710805054 +0000 UTC m=+0.421451006 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099790 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d6513e0b37ca6a7c6439fc912c8e8 SystemUUID:ec2d6513-e0b3-7ca6-a7c6-439fc912c8e8 BootID:b29e7f16-c08f-4d30-b8d8-0873d87cce40 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d7:a0:9a:a6:07 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d7:a0:9a:a6:07 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:76:90:bb:e0:38 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 28 03:16:47.712282 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.712269 2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 28 03:16:47.712392 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.712376 2555 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 28 03:16:47.713580 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.713550 2555 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 28 03:16:47.713718 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.713582 2555 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-36.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 28 03:16:47.713767 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.713728 2555 topology_manager.go:138] "Creating topology manager with none policy" May 28 03:16:47.713767 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.713736 2555 container_manager_linux.go:306] "Creating device plugin manager" May 28 03:16:47.713767 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.713743 2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 28 03:16:47.714571 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.714559 2555 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 28 03:16:47.716462 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.716451 2555 state_mem.go:36] "Initialized new in-memory state store" May 28 03:16:47.716605 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.716596 2555 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 28 03:16:47.719125 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.719112 2555 kubelet.go:491] "Attempting to sync node with API server" May 28 03:16:47.719174 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.719129 2555 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 28 03:16:47.719174 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.719142 2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 28 03:16:47.719174 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.719152 2555 kubelet.go:397] "Adding apiserver pod source" May 28 03:16:47.719174 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.719162 2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 28 03:16:47.720335 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.720320 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 28 03:16:47.720374 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.720341 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 28 03:16:47.721274 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.721251 2555 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 28 03:16:47.723434 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.723399 2555 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 28 03:16:47.727099 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.727071 2555 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 28 03:16:47.727896 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.727877 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 28 03:16:47.727987 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.727901 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 28 03:16:47.727987 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.727910 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 28 03:16:47.727987 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.727927 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 28 03:16:47.727987 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.727940 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 28 03:16:47.727987 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.727951 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 28 03:16:47.727987 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.727960 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 28 03:16:47.727987 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.727980 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 28 03:16:47.728271 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.727992 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 28 03:16:47.728271 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.728001 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 28 03:16:47.728271 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.728022 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 28 03:16:47.730146 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.730130 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 28 03:16:47.730214 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.730194 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 28 03:16:47.730214 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.730207 2555 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 28 03:16:47.733563 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.733545 2555 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 28 03:16:47.733640 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.733586 2555 server.go:1295] "Started kubelet" May 28 03:16:47.733716 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.733686 2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 28 03:16:47.733787 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.733742 2555 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 28 03:16:47.733833 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.733810 2555 server_v1.go:47] "podresources" method="list" useActivePods=true May 28 03:16:47.734329 ip-10-0-141-36 systemd[1]: Started Kubernetes Kubelet. May 28 03:16:47.735087 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.734933 2555 server.go:317] "Adding debug handlers to kubelet server" May 28 03:16:47.735087 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.734990 2555 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 28 03:16:47.738364 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.738340 2555 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-36.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 28 03:16:47.738364 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.738338 2555 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 28 03:16:47.739738 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.739717 2555 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-36.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 28 03:16:47.742138 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.742081 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 28 03:16:47.742223 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.742142 2555 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 28 03:16:47.742366 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.741324 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-36.ec2.internal.18b39c5e921abde3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-36.ec2.internal,UID:ip-10-0-141-36.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-36.ec2.internal,},FirstTimestamp:2026-05-28 03:16:47.733562851 +0000 UTC m=+0.444208804,LastTimestamp:2026-05-28 03:16:47.733562851 +0000 UTC m=+0.444208804,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-36.ec2.internal,}" May 28 03:16:47.742562 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.742543 2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 28 03:16:47.742656 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.742604 2555 volume_manager.go:295] "The desired_state_of_world populator starts" May 28 03:16:47.742656 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.742614 2555 volume_manager.go:297] "Starting Kubelet Volume Manager" May 28 03:16:47.742753 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.742737 2555 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 28 03:16:47.742815 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.742803 2555 reconstruct.go:97] "Volume reconstruction finished" May 28 03:16:47.742846 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.742817 2555 reconciler.go:26] "Reconciler: start to sync state" May 28 03:16:47.743420 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.743392 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:47.744042 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.744024 2555 factory.go:153] Registering CRI-O factory May 28 03:16:47.744042 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.744045 2555 factory.go:223] Registration of the crio container factory successfully May 28 03:16:47.744187 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.744102 2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 28 03:16:47.744187 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.744109 2555 factory.go:55] Registering systemd factory May 28 03:16:47.744187 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.744115 2555 factory.go:223] Registration of the systemd container factory successfully May 28 03:16:47.744187 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.744128 2555 factory.go:103] Registering Raw factory May 28 03:16:47.744187 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.744138 2555 manager.go:1196] Started watching for new ooms in manager May 28 03:16:47.744529 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.744517 2555 manager.go:319] Starting recovery of all containers May 28 03:16:47.753071 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.753040 2555 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-36.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" May 28 03:16:47.753163 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.753133 2555 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" May 28 03:16:47.754100 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.754081 2555 manager.go:324] Recovery completed May 28 03:16:47.759598 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.759581 2555 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 03:16:47.761321 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.761301 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasSufficientMemory" May 28 03:16:47.761390 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.761333 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasNoDiskPressure" May 28 03:16:47.761390 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.761343 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasSufficientPID" May 28 03:16:47.761902 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.761886 2555 cpu_manager.go:222] "Starting CPU manager" policy="none" May 28 03:16:47.761902 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.761900 2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 28 03:16:47.762007 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.761925 2555 state_mem.go:36] "Initialized new in-memory state store" May 28 03:16:47.764071 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.764058 2555 policy_none.go:49] "None policy: Start" May 28 03:16:47.764108 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.764075 2555 memory_manager.go:186] "Starting memorymanager" policy="None" May 28 03:16:47.764108 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.764086 2555 state_mem.go:35] "Initializing new in-memory state store" May 28 03:16:47.768625 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.768548 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-36.ec2.internal.18b39c5e93c249f6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-36.ec2.internal,UID:ip-10-0-141-36.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-36.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-36.ec2.internal,},FirstTimestamp:2026-05-28 03:16:47.761320438 +0000 UTC m=+0.471966390,LastTimestamp:2026-05-28 03:16:47.761320438 +0000 UTC m=+0.471966390,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-36.ec2.internal,}" May 28 03:16:47.777069 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.776987 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-36.ec2.internal.18b39c5e93c28cec default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-36.ec2.internal,UID:ip-10-0-141-36.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-36.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-36.ec2.internal,},FirstTimestamp:2026-05-28 03:16:47.76133758 +0000 UTC m=+0.471983532,LastTimestamp:2026-05-28 03:16:47.76133758 +0000 UTC m=+0.471983532,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-36.ec2.internal,}" May 28 03:16:47.786163 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.786090 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-36.ec2.internal.18b39c5e93c2b44c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-36.ec2.internal,UID:ip-10-0-141-36.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-36.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-36.ec2.internal,},FirstTimestamp:2026-05-28 03:16:47.76134766 +0000 UTC m=+0.471993613,LastTimestamp:2026-05-28 03:16:47.76134766 +0000 UTC m=+0.471993613,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-36.ec2.internal,}" May 28 03:16:47.797366 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.797155 2555 manager.go:341] "Starting Device Plugin manager" May 28 03:16:47.797503 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.797405 2555 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 28 03:16:47.797503 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.797421 2555 server.go:85] "Starting device plugin registration server" May 28 03:16:47.797723 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.797710 2555 eviction_manager.go:189] "Eviction manager: starting control loop" May 28 03:16:47.797789 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.797725 2555 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 28 03:16:47.797874 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.797842 2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 28 03:16:47.797949 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.797935 2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 28 03:16:47.797949 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.797949 2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 28 03:16:47.798662 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.798624 2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 28 03:16:47.798771 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.798677 2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:47.812387 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.812296 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-36.ec2.internal.18b39c5e96a926d5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-36.ec2.internal,UID:ip-10-0-141-36.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-141-36.ec2.internal,},FirstTimestamp:2026-05-28 03:16:47.810004693 +0000 UTC m=+0.520650632,LastTimestamp:2026-05-28 03:16:47.810004693 +0000 UTC m=+0.520650632,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-36.ec2.internal,}" May 28 03:16:47.829460 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.829426 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9f5bx" May 28 03:16:47.841907 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.841879 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9f5bx" May 28 03:16:47.872401 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.872365 2555 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 28 03:16:47.873640 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.873614 2555 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 28 03:16:47.873640 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.873635 2555 status_manager.go:230] "Starting to sync pod status with apiserver" May 28 03:16:47.873788 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.873656 2555 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 28 03:16:47.873788 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.873665 2555 kubelet.go:2451] "Starting kubelet main sync loop" May 28 03:16:47.873788 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.873707 2555 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 28 03:16:47.893858 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.893787 2555 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 28 03:16:47.899209 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.899198 2555 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 03:16:47.900000 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.899984 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasSufficientMemory" May 28 03:16:47.900095 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.900020 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasNoDiskPressure" May 28 03:16:47.900095 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.900031 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasSufficientPID" May 28 03:16:47.900095 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.900056 2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-36.ec2.internal" May 28 03:16:47.917970 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.917944 2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-36.ec2.internal" May 28 03:16:47.917970 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.917968 2555 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-36.ec2.internal\": node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:47.974385 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.974332 2555 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-36.ec2.internal"] May 28 03:16:47.974588 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.974436 2555 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 03:16:47.976542 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.976523 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasSufficientMemory" May 28 03:16:47.976650 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.976559 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasNoDiskPressure" May 28 03:16:47.976650 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.976573 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasSufficientPID" May 28 03:16:47.977761 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.977745 2555 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 03:16:47.977931 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.977915 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" May 28 03:16:47.977981 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.977945 2555 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 03:16:47.978599 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.978584 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasSufficientMemory" May 28 03:16:47.978599 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.978590 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasSufficientMemory" May 28 03:16:47.978736 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.978613 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasNoDiskPressure" May 28 03:16:47.978736 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.978617 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasNoDiskPressure" May 28 03:16:47.978736 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.978625 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasSufficientPID" May 28 03:16:47.978736 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.978631 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasSufficientPID" May 28 03:16:47.979812 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.979797 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-36.ec2.internal" May 28 03:16:47.979886 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.979826 2555 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 03:16:47.980543 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.980529 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasSufficientMemory" May 28 03:16:47.980629 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.980553 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasNoDiskPressure" May 28 03:16:47.980629 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:47.980567 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeHasSufficientPID" May 28 03:16:47.997700 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:47.997668 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:48.009327 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:48.009306 2555 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-36.ec2.internal\" not found" node="ip-10-0-141-36.ec2.internal" May 28 03:16:48.012821 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:48.012803 2555 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-36.ec2.internal\" not found" node="ip-10-0-141-36.ec2.internal" May 28 03:16:48.044170 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.044146 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0c80009b4a72c4b49cdec4619a368cbf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal\" (UID: \"0c80009b4a72c4b49cdec4619a368cbf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" May 28 03:16:48.044308 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.044180 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c80009b4a72c4b49cdec4619a368cbf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal\" (UID: \"0c80009b4a72c4b49cdec4619a368cbf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" May 28 03:16:48.044308 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.044207 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5f5996663d36d1c198c700af5357120a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-36.ec2.internal\" (UID: \"5f5996663d36d1c198c700af5357120a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-36.ec2.internal" May 28 03:16:48.098666 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:48.098629 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:48.144533 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.144444 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0c80009b4a72c4b49cdec4619a368cbf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal\" (UID: \"0c80009b4a72c4b49cdec4619a368cbf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" May 28 03:16:48.144533 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.144484 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c80009b4a72c4b49cdec4619a368cbf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal\" (UID: \"0c80009b4a72c4b49cdec4619a368cbf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" May 28 03:16:48.144533 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.144526 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5f5996663d36d1c198c700af5357120a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-36.ec2.internal\" (UID: \"5f5996663d36d1c198c700af5357120a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-36.ec2.internal" May 28 03:16:48.144717 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.144546 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0c80009b4a72c4b49cdec4619a368cbf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal\" (UID: \"0c80009b4a72c4b49cdec4619a368cbf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" May 28 03:16:48.144717 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.144565 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5f5996663d36d1c198c700af5357120a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-36.ec2.internal\" (UID: \"5f5996663d36d1c198c700af5357120a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-36.ec2.internal" May 28 03:16:48.144717 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.144605 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c80009b4a72c4b49cdec4619a368cbf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal\" (UID: \"0c80009b4a72c4b49cdec4619a368cbf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" May 28 03:16:48.199052 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:48.199019 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:48.300033 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:48.299999 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:48.311737 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.311712 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" May 28 03:16:48.315451 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.315435 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-36.ec2.internal" May 28 03:16:48.400511 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:48.400402 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:48.500892 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:48.500860 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:48.601423 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:48.601394 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:48.651999 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.651913 2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 28 03:16:48.652435 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.652113 2555 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 28 03:16:48.702231 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:48.702197 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:48.742328 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.742297 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 28 03:16:48.802578 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:48.802550 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:48.838555 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.838534 2555 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 28 03:16:48.844227 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.844184 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 03:11:47 +0000 UTC" deadline="2027-11-27 14:02:08.341006667 +0000 UTC" May 28 03:16:48.844306 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.844229 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13162h45m19.496782717s" May 28 03:16:48.845952 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:48.845927 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c80009b4a72c4b49cdec4619a368cbf.slice/crio-9e41a3c67c15e5952bfc3562882659eecc3df759f4af70a725a7c3992dcf8ca8 WatchSource:0}: Error finding container 9e41a3c67c15e5952bfc3562882659eecc3df759f4af70a725a7c3992dcf8ca8: Status 404 returned error can't find the container with id 9e41a3c67c15e5952bfc3562882659eecc3df759f4af70a725a7c3992dcf8ca8 May 28 03:16:48.846216 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:48.846194 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5996663d36d1c198c700af5357120a.slice/crio-771c6ac17ef1f9aa3833be476a80c5dce694d280fd9af38503272941c4dfe753 WatchSource:0}: Error finding container 771c6ac17ef1f9aa3833be476a80c5dce694d280fd9af38503272941c4dfe753: Status 404 returned error can't find the container with id 771c6ac17ef1f9aa3833be476a80c5dce694d280fd9af38503272941c4dfe753 May 28 03:16:48.850072 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.850054 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 03:16:48.876159 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.876108 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" event={"ID":"0c80009b4a72c4b49cdec4619a368cbf","Type":"ContainerStarted","Data":"9e41a3c67c15e5952bfc3562882659eecc3df759f4af70a725a7c3992dcf8ca8"} May 28 03:16:48.876913 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:48.876892 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-36.ec2.internal" event={"ID":"5f5996663d36d1c198c700af5357120a","Type":"ContainerStarted","Data":"771c6ac17ef1f9aa3833be476a80c5dce694d280fd9af38503272941c4dfe753"} May 28 03:16:48.903136 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:48.903073 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:49.003568 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:49.003537 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:49.104267 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:49.104238 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:49.110083 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.110065 2555 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 28 03:16:49.205098 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:49.205007 2555 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-36.ec2.internal\" not found" May 28 03:16:49.205292 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.205274 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-49cmp" May 28 03:16:49.213009 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.212984 2555 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 28 03:16:49.251126 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.251094 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-49cmp" May 28 03:16:49.286398 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.285872 2555 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 28 03:16:49.343967 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.343924 2555 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" May 28 03:16:49.427748 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.427708 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 28 03:16:49.428859 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.428836 2555 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-36.ec2.internal" May 28 03:16:49.487441 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.487359 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 28 03:16:49.719810 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.719781 2555 apiserver.go:52] "Watching apiserver" May 28 03:16:49.721720 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.721694 2555 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 28 03:16:49.749979 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.749898 2555 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 28 03:16:49.750324 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.750301 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-5r778","openshift-multus/multus-7zh5q","openshift-network-diagnostics/network-check-target-r4lqr","openshift-network-operator/iptables-alerter-25czr","openshift-ovn-kubernetes/ovnkube-node-mqxrv","kube-system/global-pull-secret-syncer-pkkqh","kube-system/kube-apiserver-proxy-ip-10-0-141-36.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74","openshift-dns/node-resolver-gbftm","openshift-image-registry/node-ca-zx6tr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal","openshift-multus/multus-additional-cni-plugins-4hk25","openshift-multus/network-metrics-daemon-mkdkv","kube-system/konnectivity-agent-lp9sw"] May 28 03:16:49.751772 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.751751 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gbftm" May 28 03:16:49.752886 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.752868 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.754422 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.754091 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zx6tr" May 28 03:16:49.755452 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.755396 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-25czr" May 28 03:16:49.757157 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.756848 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 28 03:16:49.757157 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.757018 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 28 03:16:49.757157 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.757153 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 28 03:16:49.757325 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.757175 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lnw94\"" May 28 03:16:49.757325 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.757205 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 28 03:16:49.757325 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.757153 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 28 03:16:49.757325 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.757268 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 28 03:16:49.757475 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.757364 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sz447\"" May 28 03:16:49.757475 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.757456 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 28 03:16:49.758063 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.757855 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 28 03:16:49.758063 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.757897 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 28 03:16:49.758063 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.757920 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 28 03:16:49.758063 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.757857 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-95ml4\"" May 28 03:16:49.758317 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.758300 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.758378 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.758336 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.759660 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.759640 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:16:49.759755 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:49.759725 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:16:49.761183 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.761006 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 28 03:16:49.762300 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.762286 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.762443 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.762408 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7zh5q" May 28 03:16:49.763564 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.763546 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:49.763640 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:49.763609 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:16:49.764980 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.764959 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lp9sw" May 28 03:16:49.766439 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.766413 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:49.766548 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:49.766516 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:16:49.766750 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.766736 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 28 03:16:49.767518 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.767485 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 28 03:16:49.767602 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.767517 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 28 03:16:49.767602 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.767568 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-px2hd\"" May 28 03:16:49.767602 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.767574 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 28 03:16:49.767602 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.767503 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 28 03:16:49.767828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.767485 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 28 03:16:49.768026 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.767997 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 28 03:16:49.768026 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.768012 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 28 03:16:49.768166 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.768010 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pvh9f\"" May 28 03:16:49.769125 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.769106 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 28 03:16:49.769221 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.769108 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 28 03:16:49.769406 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.769389 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 28 03:16:49.769482 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.769395 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 28 03:16:49.777153 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.777135 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 28 03:16:49.777241 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.777196 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w86zq\"" May 28 03:16:49.777725 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.777575 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rfj8z\"" May 28 03:16:49.778952 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.778929 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 28 03:16:49.779041 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.778989 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xw529\"" May 28 03:16:49.779403 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.779383 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 28 03:16:49.779665 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.779639 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5cj67\"" May 28 03:16:49.779758 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.779428 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 28 03:16:49.843791 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.843764 2555 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 28 03:16:49.853940 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.853905 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-systemd-units\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.853940 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.853945 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-node-log\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.854149 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.853964 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-sys\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.854149 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.853988 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-run-netns\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.854149 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854049 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-var-lib-cni-bin\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.854149 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854076 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-var-lib-cni-multus\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.854149 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854103 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-run-netns\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.854149 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854128 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-run-systemd\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.854149 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854146 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-var-lib-kubelet\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.854434 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854167 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-multus-conf-dir\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.854434 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854193 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:49.854434 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854213 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgbwn\" (UniqueName: \"kubernetes.io/projected/9d2867de-afec-49ff-be90-2680f09ddc2e-kube-api-access-dgbwn\") pod \"node-resolver-gbftm\" (UID: \"9d2867de-afec-49ff-be90-2680f09ddc2e\") " pod="openshift-dns/node-resolver-gbftm" May 28 03:16:49.854434 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854229 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-os-release\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.854434 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854268 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-slash\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.854434 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854290 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-etc-openvswitch\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.854434 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854338 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-ovn-node-metrics-cert\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.854434 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854377 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-ovnkube-script-lib\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.854434 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854403 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-modprobe-d\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.854434 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854428 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-run-multus-certs\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854456 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d2867de-afec-49ff-be90-2680f09ddc2e-hosts-file\") pod \"node-resolver-gbftm\" (UID: \"9d2867de-afec-49ff-be90-2680f09ddc2e\") " pod="openshift-dns/node-resolver-gbftm" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854479 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-kubernetes\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854518 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/06e44704-03d7-4840-a298-191be4e1b255-kubelet-config\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854556 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/06e44704-03d7-4840-a298-191be4e1b255-dbus\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854587 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-cnibin\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854617 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-log-socket\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854638 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-sysctl-d\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854661 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-system-cni-dir\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854700 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce5ad371-6b1d-464c-8aa8-6b1b0339de53-agent-certs\") pod \"konnectivity-agent-lp9sw\" (UID: \"ce5ad371-6b1d-464c-8aa8-6b1b0339de53\") " pod="kube-system/konnectivity-agent-lp9sw" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854724 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-env-overrides\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854739 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-989q8\" (UniqueName: \"kubernetes.io/projected/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-kube-api-access-989q8\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854760 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-multus-socket-dir-parent\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854776 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.854813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854804 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8txlw\" (UniqueName: \"kubernetes.io/projected/900e522d-e9a0-4344-ad62-c0518eb0cba7-kube-api-access-8txlw\") pod \"node-ca-zx6tr\" (UID: \"900e522d-e9a0-4344-ad62-c0518eb0cba7\") " pod="openshift-image-registry/node-ca-zx6tr" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854829 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mphw\" (UniqueName: \"kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw\") pod \"network-check-target-r4lqr\" (UID: \"0bfd38fd-ac06-4fc8-99e7-45f520c80876\") " pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854851 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96b96\" (UniqueName: \"kubernetes.io/projected/f8725dbf-86ef-41e7-845a-5b81d2343b2b-kube-api-access-96b96\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854869 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-run-openvswitch\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854886 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-cni-bin\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854929 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-ovnkube-config\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854956 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.854981 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-registration-dir\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855009 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-sysconfig\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855052 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/900e522d-e9a0-4344-ad62-c0518eb0cba7-serviceca\") pod \"node-ca-zx6tr\" (UID: \"900e522d-e9a0-4344-ad62-c0518eb0cba7\") " pod="openshift-image-registry/node-ca-zx6tr" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855104 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-run\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855128 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-multus-cni-dir\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855144 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-etc-kubernetes\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855169 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1eb948b1-d6ec-44ee-b86b-e12777a715c8-iptables-alerter-script\") pod \"iptables-alerter-25czr\" (UID: \"1eb948b1-d6ec-44ee-b86b-e12777a715c8\") " pod="openshift-network-operator/iptables-alerter-25czr" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855194 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-cni-netd\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855216 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-os-release\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.855330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855235 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-run-k8s-cni-cncf-io\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855258 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7a4380c-a918-4aed-ab01-ab654612890d-multus-daemon-config\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855319 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-system-cni-dir\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855356 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2lw5\" (UniqueName: \"kubernetes.io/projected/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-kube-api-access-b2lw5\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855386 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855419 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-sys-fs\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855437 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-lib-modules\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855468 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-cnibin\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855518 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855540 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kmtb\" (UniqueName: \"kubernetes.io/projected/dcbf7d2f-6d96-486a-87f4-2de98f158eca-kube-api-access-8kmtb\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855570 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-systemd\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855593 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/545c9513-745a-464c-aef6-845f2ed5f965-tmp\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855621 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7a4380c-a918-4aed-ab01-ab654612890d-cni-binary-copy\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855638 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/900e522d-e9a0-4344-ad62-c0518eb0cba7-host\") pod \"node-ca-zx6tr\" (UID: \"900e522d-e9a0-4344-ad62-c0518eb0cba7\") " pod="openshift-image-registry/node-ca-zx6tr" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855653 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1eb948b1-d6ec-44ee-b86b-e12777a715c8-host-slash\") pod \"iptables-alerter-25czr\" (UID: \"1eb948b1-d6ec-44ee-b86b-e12777a715c8\") " pod="openshift-network-operator/iptables-alerter-25czr" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855684 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce5ad371-6b1d-464c-8aa8-6b1b0339de53-konnectivity-ca\") pod \"konnectivity-agent-lp9sw\" (UID: \"ce5ad371-6b1d-464c-8aa8-6b1b0339de53\") " pod="kube-system/konnectivity-agent-lp9sw" May 28 03:16:49.856066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855711 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-kubelet\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855734 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-var-lib-openvswitch\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855756 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-host\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855786 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/545c9513-745a-464c-aef6-845f2ed5f965-etc-tuned\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855808 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855834 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-etc-selinux\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855854 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-var-lib-kubelet\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855876 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpcgf\" (UniqueName: \"kubernetes.io/projected/545c9513-745a-464c-aef6-845f2ed5f965-kube-api-access-bpcgf\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855901 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-hostroot\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855921 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855947 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-run-ovn\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855963 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-sysctl-conf\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.855983 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6442q\" (UniqueName: \"kubernetes.io/projected/f7a4380c-a918-4aed-ab01-ab654612890d-kube-api-access-6442q\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.856007 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d2867de-afec-49ff-be90-2680f09ddc2e-tmp-dir\") pod \"node-resolver-gbftm\" (UID: \"9d2867de-afec-49ff-be90-2680f09ddc2e\") " pod="openshift-dns/node-resolver-gbftm" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.856040 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.856060 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql4n2\" (UniqueName: \"kubernetes.io/projected/1eb948b1-d6ec-44ee-b86b-e12777a715c8-kube-api-access-ql4n2\") pod \"iptables-alerter-25czr\" (UID: \"1eb948b1-d6ec-44ee-b86b-e12777a715c8\") " pod="openshift-network-operator/iptables-alerter-25czr" May 28 03:16:49.856639 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.856074 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-run-ovn-kubernetes\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.857131 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.856113 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-socket-dir\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.857131 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.856159 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-device-dir\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.956618 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956582 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-96b96\" (UniqueName: \"kubernetes.io/projected/f8725dbf-86ef-41e7-845a-5b81d2343b2b-kube-api-access-96b96\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:49.956805 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956627 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-run-openvswitch\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.956805 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956648 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-cni-bin\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.956805 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956672 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-ovnkube-config\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.956805 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956741 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-run-openvswitch\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.956805 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956745 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-cni-bin\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.956805 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956774 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956808 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-registration-dir\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956832 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-sysconfig\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956860 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/900e522d-e9a0-4344-ad62-c0518eb0cba7-serviceca\") pod \"node-ca-zx6tr\" (UID: \"900e522d-e9a0-4344-ad62-c0518eb0cba7\") " pod="openshift-image-registry/node-ca-zx6tr" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956886 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-run\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956891 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956904 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-registration-dir\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956911 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-multus-cni-dir\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956928 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-sysconfig\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956937 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-etc-kubernetes\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956967 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1eb948b1-d6ec-44ee-b86b-e12777a715c8-iptables-alerter-script\") pod \"iptables-alerter-25czr\" (UID: \"1eb948b1-d6ec-44ee-b86b-e12777a715c8\") " pod="openshift-network-operator/iptables-alerter-25czr" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.956976 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-etc-kubernetes\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957026 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-cni-netd\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957038 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-run\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957051 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-os-release\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957076 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-run-k8s-cni-cncf-io\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957085 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-cni-netd\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.957092 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957100 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7a4380c-a918-4aed-ab01-ab654612890d-multus-daemon-config\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957128 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-system-cni-dir\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957129 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-multus-cni-dir\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957157 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b2lw5\" (UniqueName: \"kubernetes.io/projected/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-kube-api-access-b2lw5\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957185 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957213 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-sys-fs\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957218 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-system-cni-dir\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957236 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-lib-modules\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957243 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/900e522d-e9a0-4344-ad62-c0518eb0cba7-serviceca\") pod \"node-ca-zx6tr\" (UID: \"900e522d-e9a0-4344-ad62-c0518eb0cba7\") " pod="openshift-image-registry/node-ca-zx6tr" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957258 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-cnibin\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957187 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-os-release\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957259 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-run-k8s-cni-cncf-io\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957290 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957310 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8kmtb\" (UniqueName: \"kubernetes.io/projected/dcbf7d2f-6d96-486a-87f4-2de98f158eca-kube-api-access-8kmtb\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957309 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-sys-fs\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957315 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-ovnkube-config\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957363 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-cnibin\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.957865 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957387 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-systemd\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957438 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-lib-modules\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957438 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-systemd\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957481 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/545c9513-745a-464c-aef6-845f2ed5f965-tmp\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:49.957510 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957546 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7a4380c-a918-4aed-ab01-ab654612890d-cni-binary-copy\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957570 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1eb948b1-d6ec-44ee-b86b-e12777a715c8-iptables-alerter-script\") pod \"iptables-alerter-25czr\" (UID: \"1eb948b1-d6ec-44ee-b86b-e12777a715c8\") " pod="openshift-network-operator/iptables-alerter-25czr" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:49.957585 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs podName:f8725dbf-86ef-41e7-845a-5b81d2343b2b nodeName:}" failed. No retries permitted until 2026-05-28 03:16:50.457556143 +0000 UTC m=+3.168202083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs") pod "network-metrics-daemon-mkdkv" (UID: "f8725dbf-86ef-41e7-845a-5b81d2343b2b") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957600 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957719 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7a4380c-a918-4aed-ab01-ab654612890d-multus-daemon-config\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957711 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/900e522d-e9a0-4344-ad62-c0518eb0cba7-host\") pod \"node-ca-zx6tr\" (UID: \"900e522d-e9a0-4344-ad62-c0518eb0cba7\") " pod="openshift-image-registry/node-ca-zx6tr" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957759 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1eb948b1-d6ec-44ee-b86b-e12777a715c8-host-slash\") pod \"iptables-alerter-25czr\" (UID: \"1eb948b1-d6ec-44ee-b86b-e12777a715c8\") " pod="openshift-network-operator/iptables-alerter-25czr" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957785 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce5ad371-6b1d-464c-8aa8-6b1b0339de53-konnectivity-ca\") pod \"konnectivity-agent-lp9sw\" (UID: \"ce5ad371-6b1d-464c-8aa8-6b1b0339de53\") " pod="kube-system/konnectivity-agent-lp9sw" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957809 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-kubelet\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957812 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/900e522d-e9a0-4344-ad62-c0518eb0cba7-host\") pod \"node-ca-zx6tr\" (UID: \"900e522d-e9a0-4344-ad62-c0518eb0cba7\") " pod="openshift-image-registry/node-ca-zx6tr" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957830 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1eb948b1-d6ec-44ee-b86b-e12777a715c8-host-slash\") pod \"iptables-alerter-25czr\" (UID: \"1eb948b1-d6ec-44ee-b86b-e12777a715c8\") " pod="openshift-network-operator/iptables-alerter-25czr" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957856 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-kubelet\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957861 2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 28 03:16:49.958633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957927 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-var-lib-openvswitch\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957970 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-var-lib-openvswitch\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.957964 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-host\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958001 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/545c9513-745a-464c-aef6-845f2ed5f965-etc-tuned\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958028 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958034 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7a4380c-a918-4aed-ab01-ab654612890d-cni-binary-copy\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958054 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-etc-selinux\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958074 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-host\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958081 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-var-lib-kubelet\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958118 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcgf\" (UniqueName: \"kubernetes.io/projected/545c9513-745a-464c-aef6-845f2ed5f965-kube-api-access-bpcgf\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958145 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-hostroot\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958128 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-var-lib-kubelet\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958168 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958174 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958266 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-etc-selinux\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958364 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-run-ovn\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958369 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce5ad371-6b1d-464c-8aa8-6b1b0339de53-konnectivity-ca\") pod \"konnectivity-agent-lp9sw\" (UID: \"ce5ad371-6b1d-464c-8aa8-6b1b0339de53\") " pod="kube-system/konnectivity-agent-lp9sw" May 28 03:16:49.959264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958390 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-sysctl-conf\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958413 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6442q\" (UniqueName: \"kubernetes.io/projected/f7a4380c-a918-4aed-ab01-ab654612890d-kube-api-access-6442q\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958428 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-hostroot\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958461 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d2867de-afec-49ff-be90-2680f09ddc2e-tmp-dir\") pod \"node-resolver-gbftm\" (UID: \"9d2867de-afec-49ff-be90-2680f09ddc2e\") " pod="openshift-dns/node-resolver-gbftm" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958468 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-run-ovn\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958532 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958555 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ql4n2\" (UniqueName: \"kubernetes.io/projected/1eb948b1-d6ec-44ee-b86b-e12777a715c8-kube-api-access-ql4n2\") pod \"iptables-alerter-25czr\" (UID: \"1eb948b1-d6ec-44ee-b86b-e12777a715c8\") " pod="openshift-network-operator/iptables-alerter-25czr" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958605 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-sysctl-conf\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958708 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-run-ovn-kubernetes\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958737 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-socket-dir\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958736 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958775 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-device-dir\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958830 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-device-dir\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958838 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dcbf7d2f-6d96-486a-87f4-2de98f158eca-socket-dir\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958860 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-systemd-units\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958875 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-run-ovn-kubernetes\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958919 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-node-log\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.960033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958939 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-sys\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958973 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.958979 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-run-netns\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959004 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-var-lib-cni-bin\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959022 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-systemd-units\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959032 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-var-lib-cni-multus\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959073 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d2867de-afec-49ff-be90-2680f09ddc2e-tmp-dir\") pod \"node-resolver-gbftm\" (UID: \"9d2867de-afec-49ff-be90-2680f09ddc2e\") " pod="openshift-dns/node-resolver-gbftm" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959079 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-var-lib-cni-bin\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959077 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-run-netns\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959117 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-node-log\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959122 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-var-lib-cni-multus\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959121 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-sys\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959156 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-run-netns\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959183 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-run-systemd\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959209 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-var-lib-kubelet\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959213 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-run-netns\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959234 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-multus-conf-dir\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959252 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-run-systemd\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.960828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959256 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959289 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-var-lib-kubelet\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959298 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dgbwn\" (UniqueName: \"kubernetes.io/projected/9d2867de-afec-49ff-be90-2680f09ddc2e-kube-api-access-dgbwn\") pod \"node-resolver-gbftm\" (UID: \"9d2867de-afec-49ff-be90-2680f09ddc2e\") " pod="openshift-dns/node-resolver-gbftm" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959300 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-multus-conf-dir\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:49.959316 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:49.959369 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret podName:06e44704-03d7-4840-a298-191be4e1b255 nodeName:}" failed. No retries permitted until 2026-05-28 03:16:50.459354825 +0000 UTC m=+3.170000764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret") pod "global-pull-secret-syncer-pkkqh" (UID: "06e44704-03d7-4840-a298-191be4e1b255") : object "kube-system"/"original-pull-secret" not registered May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959404 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-os-release\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959433 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-slash\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959457 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-etc-openvswitch\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959478 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-ovn-node-metrics-cert\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959524 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-os-release\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959543 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-host-slash\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959556 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-etc-openvswitch\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959587 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-ovnkube-script-lib\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959619 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-modprobe-d\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959644 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-run-multus-certs\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959687 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-host-run-multus-certs\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.961589 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959716 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d2867de-afec-49ff-be90-2680f09ddc2e-hosts-file\") pod \"node-resolver-gbftm\" (UID: \"9d2867de-afec-49ff-be90-2680f09ddc2e\") " pod="openshift-dns/node-resolver-gbftm" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959742 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-kubernetes\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959753 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-modprobe-d\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959768 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/06e44704-03d7-4840-a298-191be4e1b255-kubelet-config\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959796 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/06e44704-03d7-4840-a298-191be4e1b255-dbus\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959808 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d2867de-afec-49ff-be90-2680f09ddc2e-hosts-file\") pod \"node-resolver-gbftm\" (UID: \"9d2867de-afec-49ff-be90-2680f09ddc2e\") " pod="openshift-dns/node-resolver-gbftm" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959821 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-cnibin\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959833 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-kubernetes\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959847 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-log-socket\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959854 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/06e44704-03d7-4840-a298-191be4e1b255-kubelet-config\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959867 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/06e44704-03d7-4840-a298-191be4e1b255-dbus\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959872 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-sysctl-d\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959883 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-cnibin\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959907 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-log-socket\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959910 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-system-cni-dir\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959938 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-system-cni-dir\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959946 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce5ad371-6b1d-464c-8aa8-6b1b0339de53-agent-certs\") pod \"konnectivity-agent-lp9sw\" (UID: \"ce5ad371-6b1d-464c-8aa8-6b1b0339de53\") " pod="kube-system/konnectivity-agent-lp9sw" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959962 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/545c9513-745a-464c-aef6-845f2ed5f965-etc-sysctl-d\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.962263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.959973 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-env-overrides\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.962956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.960000 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-989q8\" (UniqueName: \"kubernetes.io/projected/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-kube-api-access-989q8\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.962956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.960033 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-multus-socket-dir-parent\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.962956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.960060 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.962956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.960084 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8txlw\" (UniqueName: \"kubernetes.io/projected/900e522d-e9a0-4344-ad62-c0518eb0cba7-kube-api-access-8txlw\") pod \"node-ca-zx6tr\" (UID: \"900e522d-e9a0-4344-ad62-c0518eb0cba7\") " pod="openshift-image-registry/node-ca-zx6tr" May 28 03:16:49.962956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.960109 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5mphw\" (UniqueName: \"kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw\") pod \"network-check-target-r4lqr\" (UID: \"0bfd38fd-ac06-4fc8-99e7-45f520c80876\") " pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:16:49.962956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.960144 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-ovnkube-script-lib\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.962956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.960364 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7a4380c-a918-4aed-ab01-ab654612890d-multus-socket-dir-parent\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:49.962956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.960379 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-env-overrides\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.962956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.960570 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-cni-binary-copy\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:49.962956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.961769 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/545c9513-745a-464c-aef6-845f2ed5f965-tmp\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.962956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.962007 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/545c9513-745a-464c-aef6-845f2ed5f965-etc-tuned\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:49.962956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.962626 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce5ad371-6b1d-464c-8aa8-6b1b0339de53-agent-certs\") pod \"konnectivity-agent-lp9sw\" (UID: \"ce5ad371-6b1d-464c-8aa8-6b1b0339de53\") " pod="kube-system/konnectivity-agent-lp9sw" May 28 03:16:49.962956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.962631 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-ovn-node-metrics-cert\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:49.985826 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.985787 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96b96\" (UniqueName: \"kubernetes.io/projected/f8725dbf-86ef-41e7-845a-5b81d2343b2b-kube-api-access-96b96\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:49.990309 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.990271 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8txlw\" (UniqueName: \"kubernetes.io/projected/900e522d-e9a0-4344-ad62-c0518eb0cba7-kube-api-access-8txlw\") pod \"node-ca-zx6tr\" (UID: \"900e522d-e9a0-4344-ad62-c0518eb0cba7\") " pod="openshift-image-registry/node-ca-zx6tr" May 28 03:16:49.992252 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:49.992227 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6442q\" (UniqueName: \"kubernetes.io/projected/f7a4380c-a918-4aed-ab01-ab654612890d-kube-api-access-6442q\") pod \"multus-7zh5q\" (UID: \"f7a4380c-a918-4aed-ab01-ab654612890d\") " pod="openshift-multus/multus-7zh5q" May 28 03:16:50.005120 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.002198 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-989q8\" (UniqueName: \"kubernetes.io/projected/ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720-kube-api-access-989q8\") pod \"ovnkube-node-mqxrv\" (UID: \"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:50.005120 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:50.002553 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 03:16:50.005120 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:50.002576 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 03:16:50.005120 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:50.002591 2555 projected.go:194] Error preparing data for projected volume kube-api-access-5mphw for pod openshift-network-diagnostics/network-check-target-r4lqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:16:50.005120 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:50.002680 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw podName:0bfd38fd-ac06-4fc8-99e7-45f520c80876 nodeName:}" failed. No retries permitted until 2026-05-28 03:16:50.502653021 +0000 UTC m=+3.213298978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5mphw" (UniqueName: "kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw") pod "network-check-target-r4lqr" (UID: "0bfd38fd-ac06-4fc8-99e7-45f520c80876") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:16:50.005120 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.003967 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgbwn\" (UniqueName: \"kubernetes.io/projected/9d2867de-afec-49ff-be90-2680f09ddc2e-kube-api-access-dgbwn\") pod \"node-resolver-gbftm\" (UID: \"9d2867de-afec-49ff-be90-2680f09ddc2e\") " pod="openshift-dns/node-resolver-gbftm" May 28 03:16:50.005120 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.004248 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpcgf\" (UniqueName: \"kubernetes.io/projected/545c9513-745a-464c-aef6-845f2ed5f965-kube-api-access-bpcgf\") pod \"tuned-5r778\" (UID: \"545c9513-745a-464c-aef6-845f2ed5f965\") " pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:50.006000 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.005978 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2lw5\" (UniqueName: \"kubernetes.io/projected/d3e7c7f4-9f55-4d7e-9f01-7025793968a6-kube-api-access-b2lw5\") pod \"multus-additional-cni-plugins-4hk25\" (UID: \"d3e7c7f4-9f55-4d7e-9f01-7025793968a6\") " pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:50.006161 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.006137 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql4n2\" (UniqueName: \"kubernetes.io/projected/1eb948b1-d6ec-44ee-b86b-e12777a715c8-kube-api-access-ql4n2\") pod \"iptables-alerter-25czr\" (UID: \"1eb948b1-d6ec-44ee-b86b-e12777a715c8\") " pod="openshift-network-operator/iptables-alerter-25czr" May 28 03:16:50.009640 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.009616 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kmtb\" (UniqueName: \"kubernetes.io/projected/dcbf7d2f-6d96-486a-87f4-2de98f158eca-kube-api-access-8kmtb\") pod \"aws-ebs-csi-driver-node-crt74\" (UID: \"dcbf7d2f-6d96-486a-87f4-2de98f158eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:50.065852 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.065821 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gbftm" May 28 03:16:50.072670 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.072647 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4hk25" May 28 03:16:50.082284 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.082256 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zx6tr" May 28 03:16:50.088005 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.087972 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-25czr" May 28 03:16:50.093797 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.093769 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:16:50.102211 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.102190 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" May 28 03:16:50.106850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.106828 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5r778" May 28 03:16:50.113040 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.113020 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7zh5q" May 28 03:16:50.118102 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.118080 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lp9sw" May 28 03:16:50.252268 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.252230 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 03:11:49 +0000 UTC" deadline="2028-01-30 18:02:27.608841398 +0000 UTC" May 28 03:16:50.252268 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.252272 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14702h45m37.35658352s" May 28 03:16:50.463192 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.463101 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:50.463192 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.463172 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:50.463402 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:50.463272 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:16:50.463402 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:50.463274 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 03:16:50.463402 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:50.463334 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs podName:f8725dbf-86ef-41e7-845a-5b81d2343b2b nodeName:}" failed. No retries permitted until 2026-05-28 03:16:51.463317647 +0000 UTC m=+4.173963589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs") pod "network-metrics-daemon-mkdkv" (UID: "f8725dbf-86ef-41e7-845a-5b81d2343b2b") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:16:50.463402 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:50.463348 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret podName:06e44704-03d7-4840-a298-191be4e1b255 nodeName:}" failed. No retries permitted until 2026-05-28 03:16:51.463342355 +0000 UTC m=+4.173988294 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret") pod "global-pull-secret-syncer-pkkqh" (UID: "06e44704-03d7-4840-a298-191be4e1b255") : object "kube-system"/"original-pull-secret" not registered May 28 03:16:50.564211 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.564175 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5mphw\" (UniqueName: \"kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw\") pod \"network-check-target-r4lqr\" (UID: \"0bfd38fd-ac06-4fc8-99e7-45f520c80876\") " pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:16:50.564372 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:50.564324 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 03:16:50.564372 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:50.564342 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 03:16:50.564372 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:50.564354 2555 projected.go:194] Error preparing data for projected volume kube-api-access-5mphw for pod openshift-network-diagnostics/network-check-target-r4lqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:16:50.564547 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:50.564420 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw podName:0bfd38fd-ac06-4fc8-99e7-45f520c80876 nodeName:}" failed. No retries permitted until 2026-05-28 03:16:51.564401261 +0000 UTC m=+4.275047207 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5mphw" (UniqueName: "kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw") pod "network-check-target-r4lqr" (UID: "0bfd38fd-ac06-4fc8-99e7-45f520c80876") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:16:50.584997 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:50.584965 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec36f7a6_5a79_46b3_b2c4_c9ff7ecea720.slice/crio-4619e92554b396677a71183256f9c6f635b6892cbe09e9f6fce59483c5e21011 WatchSource:0}: Error finding container 4619e92554b396677a71183256f9c6f635b6892cbe09e9f6fce59483c5e21011: Status 404 returned error can't find the container with id 4619e92554b396677a71183256f9c6f635b6892cbe09e9f6fce59483c5e21011 May 28 03:16:50.588891 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:50.588851 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2867de_afec_49ff_be90_2680f09ddc2e.slice/crio-54eaaf0a4eb2345f397ffdec66ce2a20f5ebbc6a639027e1ac8a653e9dd2dabe WatchSource:0}: Error finding container 54eaaf0a4eb2345f397ffdec66ce2a20f5ebbc6a639027e1ac8a653e9dd2dabe: Status 404 returned error can't find the container with id 54eaaf0a4eb2345f397ffdec66ce2a20f5ebbc6a639027e1ac8a653e9dd2dabe May 28 03:16:50.590284 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:50.590260 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod900e522d_e9a0_4344_ad62_c0518eb0cba7.slice/crio-478c6f032b31f5f7195eb362ed90fc2733ba2b7fce15741fcfac110c52cdd901 WatchSource:0}: Error finding container 478c6f032b31f5f7195eb362ed90fc2733ba2b7fce15741fcfac110c52cdd901: Status 404 returned error can't find the container with id 478c6f032b31f5f7195eb362ed90fc2733ba2b7fce15741fcfac110c52cdd901 May 28 03:16:50.591454 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:50.591368 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eb948b1_d6ec_44ee_b86b_e12777a715c8.slice/crio-d3c13737867d0890fec4996d7232b54dafe29a87b8777fec4f6601b77942866d WatchSource:0}: Error finding container d3c13737867d0890fec4996d7232b54dafe29a87b8777fec4f6601b77942866d: Status 404 returned error can't find the container with id d3c13737867d0890fec4996d7232b54dafe29a87b8777fec4f6601b77942866d May 28 03:16:50.591852 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:50.591804 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3e7c7f4_9f55_4d7e_9f01_7025793968a6.slice/crio-9dba67127e3ee0b8a15ece425998ebcab94345f439d0e0c5a0dd9d3694bdf099 WatchSource:0}: Error finding container 9dba67127e3ee0b8a15ece425998ebcab94345f439d0e0c5a0dd9d3694bdf099: Status 404 returned error can't find the container with id 9dba67127e3ee0b8a15ece425998ebcab94345f439d0e0c5a0dd9d3694bdf099 May 28 03:16:50.593723 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:50.593685 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcbf7d2f_6d96_486a_87f4_2de98f158eca.slice/crio-f93fe8620a517b73e23a3d078ba9f529564af98ec7562afec55352e84853e3b1 WatchSource:0}: Error finding container f93fe8620a517b73e23a3d078ba9f529564af98ec7562afec55352e84853e3b1: Status 404 returned error can't find the container with id f93fe8620a517b73e23a3d078ba9f529564af98ec7562afec55352e84853e3b1 May 28 03:16:50.594699 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:50.594676 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7a4380c_a918_4aed_ab01_ab654612890d.slice/crio-e472f1b4e80027f74c536b4faf8245f0fe5c577babf717b25b8f70fe6f1eee5e WatchSource:0}: Error finding container e472f1b4e80027f74c536b4faf8245f0fe5c577babf717b25b8f70fe6f1eee5e: Status 404 returned error can't find the container with id e472f1b4e80027f74c536b4faf8245f0fe5c577babf717b25b8f70fe6f1eee5e May 28 03:16:50.596956 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:50.596863 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce5ad371_6b1d_464c_8aa8_6b1b0339de53.slice/crio-f4bd109b7d75fe9934ca7a95da4a12063d28434a428544ecf8e22cbad8b56ee1 WatchSource:0}: Error finding container f4bd109b7d75fe9934ca7a95da4a12063d28434a428544ecf8e22cbad8b56ee1: Status 404 returned error can't find the container with id f4bd109b7d75fe9934ca7a95da4a12063d28434a428544ecf8e22cbad8b56ee1 May 28 03:16:50.599133 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:16:50.599097 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod545c9513_745a_464c_aef6_845f2ed5f965.slice/crio-a866e0d78e3e7a79a89d54ec8f3a260ec6e2d6ff261538d8a90280956a152f08 WatchSource:0}: Error finding container a866e0d78e3e7a79a89d54ec8f3a260ec6e2d6ff261538d8a90280956a152f08: Status 404 returned error can't find the container with id a866e0d78e3e7a79a89d54ec8f3a260ec6e2d6ff261538d8a90280956a152f08 May 28 03:16:50.874311 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.874279 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:16:50.874812 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:50.874400 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:16:50.880965 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.880934 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" event={"ID":"0c80009b4a72c4b49cdec4619a368cbf","Type":"ContainerStarted","Data":"504ca7add93fc2ae2ba584bb339c53ec40bfd5b456a4a51944b34f3307ce09d0"} May 28 03:16:50.882140 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.882113 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-36.ec2.internal" event={"ID":"5f5996663d36d1c198c700af5357120a","Type":"ContainerStarted","Data":"0b17f412432dbb6cc35421148c732580ee5f5a0ba9de00e18371e4e2fd7734aa"} May 28 03:16:50.884445 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.884421 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lp9sw" event={"ID":"ce5ad371-6b1d-464c-8aa8-6b1b0339de53","Type":"ContainerStarted","Data":"f4bd109b7d75fe9934ca7a95da4a12063d28434a428544ecf8e22cbad8b56ee1"} May 28 03:16:50.885538 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.885516 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7zh5q" event={"ID":"f7a4380c-a918-4aed-ab01-ab654612890d","Type":"ContainerStarted","Data":"e472f1b4e80027f74c536b4faf8245f0fe5c577babf717b25b8f70fe6f1eee5e"} May 28 03:16:50.886563 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.886520 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" event={"ID":"dcbf7d2f-6d96-486a-87f4-2de98f158eca","Type":"ContainerStarted","Data":"f93fe8620a517b73e23a3d078ba9f529564af98ec7562afec55352e84853e3b1"} May 28 03:16:50.887912 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.887891 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zx6tr" event={"ID":"900e522d-e9a0-4344-ad62-c0518eb0cba7","Type":"ContainerStarted","Data":"478c6f032b31f5f7195eb362ed90fc2733ba2b7fce15741fcfac110c52cdd901"} May 28 03:16:50.888797 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.888769 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5r778" event={"ID":"545c9513-745a-464c-aef6-845f2ed5f965","Type":"ContainerStarted","Data":"a866e0d78e3e7a79a89d54ec8f3a260ec6e2d6ff261538d8a90280956a152f08"} May 28 03:16:50.893635 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.893605 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4hk25" event={"ID":"d3e7c7f4-9f55-4d7e-9f01-7025793968a6","Type":"ContainerStarted","Data":"9dba67127e3ee0b8a15ece425998ebcab94345f439d0e0c5a0dd9d3694bdf099"} May 28 03:16:50.894936 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.894901 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-25czr" event={"ID":"1eb948b1-d6ec-44ee-b86b-e12777a715c8","Type":"ContainerStarted","Data":"d3c13737867d0890fec4996d7232b54dafe29a87b8777fec4f6601b77942866d"} May 28 03:16:50.895818 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.895799 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gbftm" event={"ID":"9d2867de-afec-49ff-be90-2680f09ddc2e","Type":"ContainerStarted","Data":"54eaaf0a4eb2345f397ffdec66ce2a20f5ebbc6a639027e1ac8a653e9dd2dabe"} May 28 03:16:50.896727 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.896703 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" event={"ID":"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720","Type":"ContainerStarted","Data":"4619e92554b396677a71183256f9c6f635b6892cbe09e9f6fce59483c5e21011"} May 28 03:16:50.962228 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:50.962126 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-36.ec2.internal" podStartSLOduration=1.962105163 podStartE2EDuration="1.962105163s" podCreationTimestamp="2026-05-28 03:16:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 03:16:50.961538246 +0000 UTC m=+3.672184205" watchObservedRunningTime="2026-05-28 03:16:50.962105163 +0000 UTC m=+3.672751127" May 28 03:16:51.253232 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:51.253131 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 03:11:49 +0000 UTC" deadline="2028-02-22 17:40:21.13987884 +0000 UTC" May 28 03:16:51.253232 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:51.253178 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15254h23m29.886704628s" May 28 03:16:51.471888 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:51.471850 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:51.472084 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:51.471947 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:51.472154 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:51.472092 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:16:51.472203 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:51.472155 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs podName:f8725dbf-86ef-41e7-845a-5b81d2343b2b nodeName:}" failed. No retries permitted until 2026-05-28 03:16:53.472136443 +0000 UTC m=+6.182782386 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs") pod "network-metrics-daemon-mkdkv" (UID: "f8725dbf-86ef-41e7-845a-5b81d2343b2b") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:16:51.472616 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:51.472597 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 03:16:51.472712 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:51.472654 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret podName:06e44704-03d7-4840-a298-191be4e1b255 nodeName:}" failed. No retries permitted until 2026-05-28 03:16:53.472638564 +0000 UTC m=+6.183284504 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret") pod "global-pull-secret-syncer-pkkqh" (UID: "06e44704-03d7-4840-a298-191be4e1b255") : object "kube-system"/"original-pull-secret" not registered May 28 03:16:51.572742 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:51.572659 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5mphw\" (UniqueName: \"kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw\") pod \"network-check-target-r4lqr\" (UID: \"0bfd38fd-ac06-4fc8-99e7-45f520c80876\") " pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:16:51.572900 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:51.572881 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 03:16:51.572900 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:51.572899 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 03:16:51.573022 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:51.572912 2555 projected.go:194] Error preparing data for projected volume kube-api-access-5mphw for pod openshift-network-diagnostics/network-check-target-r4lqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:16:51.573022 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:51.572971 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw podName:0bfd38fd-ac06-4fc8-99e7-45f520c80876 nodeName:}" failed. No retries permitted until 2026-05-28 03:16:53.572952865 +0000 UTC m=+6.283598826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5mphw" (UniqueName: "kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw") pod "network-check-target-r4lqr" (UID: "0bfd38fd-ac06-4fc8-99e7-45f520c80876") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:16:51.876254 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:51.876163 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:51.876766 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:51.876293 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:16:51.876766 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:51.876731 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:51.876883 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:51.876841 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:16:51.921173 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:51.920254 2555 generic.go:358] "Generic (PLEG): container finished" podID="0c80009b4a72c4b49cdec4619a368cbf" containerID="504ca7add93fc2ae2ba584bb339c53ec40bfd5b456a4a51944b34f3307ce09d0" exitCode=0 May 28 03:16:51.921173 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:51.921124 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" event={"ID":"0c80009b4a72c4b49cdec4619a368cbf","Type":"ContainerDied","Data":"504ca7add93fc2ae2ba584bb339c53ec40bfd5b456a4a51944b34f3307ce09d0"} May 28 03:16:52.875330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:52.874836 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:16:52.875330 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:52.874969 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:16:52.938803 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:52.938765 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" event={"ID":"0c80009b4a72c4b49cdec4619a368cbf","Type":"ContainerStarted","Data":"fdfdcac59a28f0be501d56dfcd5aeabe7518a41cf3a112fffa5cacd9cf29cc52"} May 28 03:16:52.971796 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:52.971740 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-36.ec2.internal" podStartSLOduration=3.9717206259999998 podStartE2EDuration="3.971720626s" podCreationTimestamp="2026-05-28 03:16:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 03:16:52.971001126 +0000 UTC m=+5.681647093" watchObservedRunningTime="2026-05-28 03:16:52.971720626 +0000 UTC m=+5.682366586" May 28 03:16:53.497566 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:53.496768 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:53.497566 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:53.496845 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:53.497566 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:53.496969 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 03:16:53.497566 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:53.497033 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret podName:06e44704-03d7-4840-a298-191be4e1b255 nodeName:}" failed. No retries permitted until 2026-05-28 03:16:57.497014638 +0000 UTC m=+10.207660584 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret") pod "global-pull-secret-syncer-pkkqh" (UID: "06e44704-03d7-4840-a298-191be4e1b255") : object "kube-system"/"original-pull-secret" not registered May 28 03:16:53.497566 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:53.497460 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:16:53.497566 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:53.497526 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs podName:f8725dbf-86ef-41e7-845a-5b81d2343b2b nodeName:}" failed. No retries permitted until 2026-05-28 03:16:57.497509527 +0000 UTC m=+10.208155470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs") pod "network-metrics-daemon-mkdkv" (UID: "f8725dbf-86ef-41e7-845a-5b81d2343b2b") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:16:53.597769 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:53.597736 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5mphw\" (UniqueName: \"kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw\") pod \"network-check-target-r4lqr\" (UID: \"0bfd38fd-ac06-4fc8-99e7-45f520c80876\") " pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:16:53.597949 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:53.597892 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 03:16:53.597949 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:53.597906 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 03:16:53.597949 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:53.597914 2555 projected.go:194] Error preparing data for projected volume kube-api-access-5mphw for pod openshift-network-diagnostics/network-check-target-r4lqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:16:53.598104 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:53.597963 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw podName:0bfd38fd-ac06-4fc8-99e7-45f520c80876 nodeName:}" failed. No retries permitted until 2026-05-28 03:16:57.597946112 +0000 UTC m=+10.308592053 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5mphw" (UniqueName: "kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw") pod "network-check-target-r4lqr" (UID: "0bfd38fd-ac06-4fc8-99e7-45f520c80876") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:16:53.874222 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:53.874039 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:53.874222 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:53.874097 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:53.874222 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:53.874194 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:16:53.874471 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:53.874325 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:16:54.874680 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:54.874645 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:16:54.875078 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:54.874767 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:16:55.875322 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:55.874569 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:55.875322 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:55.874569 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:55.875322 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:55.874773 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:16:55.875322 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:55.874888 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:16:56.874991 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:56.874746 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:16:56.874991 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:56.874890 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:16:57.529753 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:57.529711 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:57.530287 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:57.529790 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:57.530287 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:57.529875 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:16:57.530287 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:57.529938 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 03:16:57.530287 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:57.529951 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs podName:f8725dbf-86ef-41e7-845a-5b81d2343b2b nodeName:}" failed. No retries permitted until 2026-05-28 03:17:05.52993084 +0000 UTC m=+18.240576782 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs") pod "network-metrics-daemon-mkdkv" (UID: "f8725dbf-86ef-41e7-845a-5b81d2343b2b") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:16:57.530287 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:57.529997 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret podName:06e44704-03d7-4840-a298-191be4e1b255 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:05.529978946 +0000 UTC m=+18.240624905 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret") pod "global-pull-secret-syncer-pkkqh" (UID: "06e44704-03d7-4840-a298-191be4e1b255") : object "kube-system"/"original-pull-secret" not registered May 28 03:16:57.630537 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:57.630236 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5mphw\" (UniqueName: \"kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw\") pod \"network-check-target-r4lqr\" (UID: \"0bfd38fd-ac06-4fc8-99e7-45f520c80876\") " pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:16:57.630537 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:57.630415 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 03:16:57.630537 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:57.630435 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 03:16:57.630537 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:57.630447 2555 projected.go:194] Error preparing data for projected volume kube-api-access-5mphw for pod openshift-network-diagnostics/network-check-target-r4lqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:16:57.630537 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:57.630522 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw podName:0bfd38fd-ac06-4fc8-99e7-45f520c80876 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:05.630504068 +0000 UTC m=+18.341150022 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5mphw" (UniqueName: "kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw") pod "network-check-target-r4lqr" (UID: "0bfd38fd-ac06-4fc8-99e7-45f520c80876") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:16:57.876079 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:57.875369 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:57.876079 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:57.875514 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:16:57.876079 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:57.875897 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:57.876079 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:57.875987 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:16:58.874287 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:58.874246 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:16:58.874866 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:58.874382 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:16:59.877531 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:59.877486 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:16:59.877531 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:16:59.877486 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:16:59.878046 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:59.877636 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:16:59.878046 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:16:59.877747 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:17:00.874773 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:00.874695 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:00.874921 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:00.874815 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:17:01.874541 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:01.874483 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:01.874541 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:01.874532 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:01.875024 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:01.874649 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:17:01.875024 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:01.874837 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:17:02.873919 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:02.873882 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:02.874106 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:02.874008 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:17:03.874291 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:03.874259 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:03.874708 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:03.874302 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:03.874708 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:03.874387 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:17:03.874708 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:03.874536 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:17:04.874181 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:04.874141 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:04.874439 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:04.874280 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:17:05.592053 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:05.592001 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:05.592251 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:05.592070 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:05.592251 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:05.592180 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 03:17:05.592251 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:05.592245 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret podName:06e44704-03d7-4840-a298-191be4e1b255 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:21.592225649 +0000 UTC m=+34.302871596 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret") pod "global-pull-secret-syncer-pkkqh" (UID: "06e44704-03d7-4840-a298-191be4e1b255") : object "kube-system"/"original-pull-secret" not registered May 28 03:17:05.592413 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:05.592181 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:17:05.592413 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:05.592326 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs podName:f8725dbf-86ef-41e7-845a-5b81d2343b2b nodeName:}" failed. No retries permitted until 2026-05-28 03:17:21.592309616 +0000 UTC m=+34.302955558 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs") pod "network-metrics-daemon-mkdkv" (UID: "f8725dbf-86ef-41e7-845a-5b81d2343b2b") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:17:05.693088 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:05.693043 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5mphw\" (UniqueName: \"kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw\") pod \"network-check-target-r4lqr\" (UID: \"0bfd38fd-ac06-4fc8-99e7-45f520c80876\") " pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:05.693259 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:05.693212 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 03:17:05.693259 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:05.693231 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 03:17:05.693259 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:05.693241 2555 projected.go:194] Error preparing data for projected volume kube-api-access-5mphw for pod openshift-network-diagnostics/network-check-target-r4lqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:17:05.693375 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:05.693303 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw podName:0bfd38fd-ac06-4fc8-99e7-45f520c80876 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:21.69328495 +0000 UTC m=+34.403930909 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5mphw" (UniqueName: "kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw") pod "network-check-target-r4lqr" (UID: "0bfd38fd-ac06-4fc8-99e7-45f520c80876") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:17:05.874737 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:05.874652 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:05.875158 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:05.874808 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:17:05.875213 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:05.875163 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:05.875327 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:05.875305 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:17:06.874036 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:06.873998 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:06.874248 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:06.874126 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:17:07.875020 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:07.874976 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:07.875510 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:07.875099 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:17:07.875510 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:07.875150 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:07.875510 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:07.875248 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:17:08.874365 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:08.874341 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:08.874478 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:08.874460 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:17:09.874948 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.874395 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:09.874948 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.874422 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:09.875826 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:09.874951 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:17:09.875826 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:09.875088 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:17:09.968502 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.968453 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5r778" event={"ID":"545c9513-745a-464c-aef6-845f2ed5f965","Type":"ContainerStarted","Data":"9860766c6ee567df3bfa56ff7767c4c02292ca6e874f30cc069e6a704408d9ab"} May 28 03:17:09.969844 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.969817 2555 generic.go:358] "Generic (PLEG): container finished" podID="d3e7c7f4-9f55-4d7e-9f01-7025793968a6" containerID="4e932c80df5512f674cddfd9edddbacdb18e1278f87b8adfba74f1b780bcd358" exitCode=0 May 28 03:17:09.969959 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.969905 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4hk25" event={"ID":"d3e7c7f4-9f55-4d7e-9f01-7025793968a6","Type":"ContainerDied","Data":"4e932c80df5512f674cddfd9edddbacdb18e1278f87b8adfba74f1b780bcd358"} May 28 03:17:09.971127 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.971102 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gbftm" event={"ID":"9d2867de-afec-49ff-be90-2680f09ddc2e","Type":"ContainerStarted","Data":"6693304cee4b9746f34f0e4582cb50426856211b9410c0537d6c0152cbd1af71"} May 28 03:17:09.974885 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.974864 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:17:09.975527 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.975504 2555 generic.go:358] "Generic (PLEG): container finished" podID="ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720" containerID="c8e9b1e5ad0d996a93ee234eaa7591b47250a2719b1ecc754647ac9baaa55dd5" exitCode=1 May 28 03:17:09.975629 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.975528 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" event={"ID":"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720","Type":"ContainerStarted","Data":"625c72177e233ae2dc7a214ecd483b5ba0551fee7ed9c0adb7d239b6dc3a6bb3"} May 28 03:17:09.975629 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.975549 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" event={"ID":"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720","Type":"ContainerStarted","Data":"929238a7ff24be2db1ab617254223292a2668a66709057b122d715811b0e6771"} May 28 03:17:09.975629 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.975558 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" event={"ID":"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720","Type":"ContainerStarted","Data":"4eeec0184037ef10e78b9c92eeeb8907b9726db1a4fe82ef9f1dcc6b895abc03"} May 28 03:17:09.975629 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.975566 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" event={"ID":"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720","Type":"ContainerStarted","Data":"8f4e33921682e05428bb97527f5e8b7b97ffdd7ede6006a5c8574a3368cb06ca"} May 28 03:17:09.975629 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.975579 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" event={"ID":"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720","Type":"ContainerDied","Data":"c8e9b1e5ad0d996a93ee234eaa7591b47250a2719b1ecc754647ac9baaa55dd5"} May 28 03:17:09.975629 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.975591 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" event={"ID":"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720","Type":"ContainerStarted","Data":"ce3b4c19f66b9a81b2ff49693b31e8ab9827c9abb359b75666f3be474c298642"} May 28 03:17:09.976727 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.976704 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lp9sw" event={"ID":"ce5ad371-6b1d-464c-8aa8-6b1b0339de53","Type":"ContainerStarted","Data":"b2be5e6ac5104de45972f3f4e0f71554a5f76b3ac66322ac45a8eba4c8d16a2b"} May 28 03:17:09.978011 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.977980 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7zh5q" event={"ID":"f7a4380c-a918-4aed-ab01-ab654612890d","Type":"ContainerStarted","Data":"f34524b70cc6518383763fb5842f5576615d643023b17c524ebe15f1cfd816ca"} May 28 03:17:09.979368 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.979345 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" event={"ID":"dcbf7d2f-6d96-486a-87f4-2de98f158eca","Type":"ContainerStarted","Data":"327ecd3735f871a274f80c6a4723c72eec5f30a2ac6b3354541c2a9e7ba55acc"} May 28 03:17:09.980805 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:09.980780 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zx6tr" event={"ID":"900e522d-e9a0-4344-ad62-c0518eb0cba7","Type":"ContainerStarted","Data":"da2b84dc909802d0bf8e30744e938379bf38337f3fd535d0d63a4a169fb2d6cd"} May 28 03:17:10.034290 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:10.034225 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5r778" podStartSLOduration=3.794616851 podStartE2EDuration="22.034209579s" podCreationTimestamp="2026-05-28 03:16:48 +0000 UTC" firstStartedPulling="2026-05-28 03:16:50.602017026 +0000 UTC m=+3.312662969" lastFinishedPulling="2026-05-28 03:17:08.841609754 +0000 UTC m=+21.552255697" observedRunningTime="2026-05-28 03:17:10.002405446 +0000 UTC m=+22.713051409" watchObservedRunningTime="2026-05-28 03:17:10.034209579 +0000 UTC m=+22.744855541" May 28 03:17:10.060066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:10.059967 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-7zh5q" podStartSLOduration=3.790566491 podStartE2EDuration="22.059949185s" podCreationTimestamp="2026-05-28 03:16:48 +0000 UTC" firstStartedPulling="2026-05-28 03:16:50.599309722 +0000 UTC m=+3.309955678" lastFinishedPulling="2026-05-28 03:17:08.868692432 +0000 UTC m=+21.579338372" observedRunningTime="2026-05-28 03:17:10.035007843 +0000 UTC m=+22.745653810" watchObservedRunningTime="2026-05-28 03:17:10.059949185 +0000 UTC m=+22.770595148" May 28 03:17:10.060627 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:10.060588 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-gbftm" podStartSLOduration=4.821710525 podStartE2EDuration="23.060579821s" podCreationTimestamp="2026-05-28 03:16:47 +0000 UTC" firstStartedPulling="2026-05-28 03:16:50.590913265 +0000 UTC m=+3.301559210" lastFinishedPulling="2026-05-28 03:17:08.829782551 +0000 UTC m=+21.540428506" observedRunningTime="2026-05-28 03:17:10.059628403 +0000 UTC m=+22.770274366" watchObservedRunningTime="2026-05-28 03:17:10.060579821 +0000 UTC m=+22.771225786" May 28 03:17:10.108180 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:10.108125 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lp9sw" podStartSLOduration=8.507743487 podStartE2EDuration="22.108113291s" podCreationTimestamp="2026-05-28 03:16:48 +0000 UTC" firstStartedPulling="2026-05-28 03:16:50.600208332 +0000 UTC m=+3.310854272" lastFinishedPulling="2026-05-28 03:17:04.200578136 +0000 UTC m=+16.911224076" observedRunningTime="2026-05-28 03:17:10.107818458 +0000 UTC m=+22.818464422" watchObservedRunningTime="2026-05-28 03:17:10.108113291 +0000 UTC m=+22.818759253" May 28 03:17:10.557098 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:10.557069 2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 28 03:17:10.810700 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:10.810546 2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T03:17:10.557090645Z","UUID":"b4b18120-5cff-47bc-964a-5617d6ac0f72","Handler":null,"Name":"","Endpoint":""} May 28 03:17:10.812471 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:10.812442 2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 28 03:17:10.812471 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:10.812475 2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 28 03:17:10.874245 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:10.874207 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:10.874404 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:10.874317 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:17:10.985218 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:10.985179 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" event={"ID":"dcbf7d2f-6d96-486a-87f4-2de98f158eca","Type":"ContainerStarted","Data":"e118686f4afbde8e4d4532648e541c1d0a794fda6bf5adff9a33ff2ad6416545"} May 28 03:17:10.986559 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:10.986528 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-25czr" event={"ID":"1eb948b1-d6ec-44ee-b86b-e12777a715c8","Type":"ContainerStarted","Data":"5df6d22e755e2d4a6efe7102f70f80ad303b38921f36cbfaa2e4fda983df9ac1"} May 28 03:17:11.056857 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:11.056806 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zx6tr" podStartSLOduration=4.81416198 podStartE2EDuration="23.056788986s" podCreationTimestamp="2026-05-28 03:16:48 +0000 UTC" firstStartedPulling="2026-05-28 03:16:50.592716126 +0000 UTC m=+3.303362070" lastFinishedPulling="2026-05-28 03:17:08.835343119 +0000 UTC m=+21.545989076" observedRunningTime="2026-05-28 03:17:10.231946423 +0000 UTC m=+22.942592385" watchObservedRunningTime="2026-05-28 03:17:11.056788986 +0000 UTC m=+23.767434994" May 28 03:17:11.874122 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:11.874077 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:11.874303 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:11.874077 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:11.874303 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:11.874199 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:17:11.874303 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:11.874263 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:17:12.874969 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:12.874769 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:12.875425 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:12.875036 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:17:12.993410 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:12.993367 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" event={"ID":"dcbf7d2f-6d96-486a-87f4-2de98f158eca","Type":"ContainerStarted","Data":"6088398ae9d83ab49012e4037f14595d5c12a83ab1585c00c4da94c3b192e92e"} May 28 03:17:12.996423 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:12.996401 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:17:12.996745 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:12.996719 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" event={"ID":"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720","Type":"ContainerStarted","Data":"4e4ca2672360b1b2cdeb722515e091ff68de000613c65c78334196a3443face4"} May 28 03:17:13.040283 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:13.040236 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-25czr" podStartSLOduration=6.797985098 podStartE2EDuration="25.040223176s" podCreationTimestamp="2026-05-28 03:16:48 +0000 UTC" firstStartedPulling="2026-05-28 03:16:50.593462568 +0000 UTC m=+3.304108511" lastFinishedPulling="2026-05-28 03:17:08.835700642 +0000 UTC m=+21.546346589" observedRunningTime="2026-05-28 03:17:11.057229703 +0000 UTC m=+23.767875664" watchObservedRunningTime="2026-05-28 03:17:13.040223176 +0000 UTC m=+25.750869159" May 28 03:17:13.040474 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:13.040455 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-crt74" podStartSLOduration=3.6901039129999997 podStartE2EDuration="25.040451868s" podCreationTimestamp="2026-05-28 03:16:48 +0000 UTC" firstStartedPulling="2026-05-28 03:16:50.59665173 +0000 UTC m=+3.307297686" lastFinishedPulling="2026-05-28 03:17:11.946999702 +0000 UTC m=+24.657645641" observedRunningTime="2026-05-28 03:17:13.040010289 +0000 UTC m=+25.750656285" watchObservedRunningTime="2026-05-28 03:17:13.040451868 +0000 UTC m=+25.751097848" May 28 03:17:13.874968 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:13.874880 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:13.875100 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:13.874890 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:13.875366 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:13.875120 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:17:13.875366 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:13.874994 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:17:14.624090 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:14.624050 2555 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lp9sw" May 28 03:17:14.624798 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:14.624775 2555 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lp9sw" May 28 03:17:14.873989 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:14.873954 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:14.874179 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:14.874057 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:17:15.003654 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:15.003626 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:17:15.004530 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:15.004008 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" event={"ID":"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720","Type":"ContainerStarted","Data":"28fe995110cf1248ae463555bacce3d49e03240b53c995c10bac31c47e9ebc4e"} May 28 03:17:15.004530 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:15.004199 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lp9sw" May 28 03:17:15.004530 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:15.004399 2555 scope.go:117] "RemoveContainer" containerID="c8e9b1e5ad0d996a93ee234eaa7591b47250a2719b1ecc754647ac9baaa55dd5" May 28 03:17:15.004788 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:15.004762 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lp9sw" May 28 03:17:15.877548 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:15.877340 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:15.877746 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:15.877356 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:15.877746 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:15.877649 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:17:15.877746 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:15.877733 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:17:16.011389 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.011364 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:17:16.011968 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.011933 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" event={"ID":"ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720","Type":"ContainerStarted","Data":"b9f84fb9a746c994ea8ebfc9828d08bff26162055dc7c96f5b53d7542dc62434"} May 28 03:17:16.012726 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.012701 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:17:16.012813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.012731 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:17:16.012813 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.012744 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:17:16.033925 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.033893 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:17:16.034803 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.034767 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:17:16.183784 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.183735 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" podStartSLOduration=9.506896404999999 podStartE2EDuration="28.183572666s" podCreationTimestamp="2026-05-28 03:16:48 +0000 UTC" firstStartedPulling="2026-05-28 03:16:50.587446773 +0000 UTC m=+3.298092713" lastFinishedPulling="2026-05-28 03:17:09.264123034 +0000 UTC m=+21.974768974" observedRunningTime="2026-05-28 03:17:16.116965962 +0000 UTC m=+28.827611950" watchObservedRunningTime="2026-05-28 03:17:16.183572666 +0000 UTC m=+28.894218629" May 28 03:17:16.329645 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.329610 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pkkqh"] May 28 03:17:16.329841 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.329734 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:16.329914 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:16.329849 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:17:16.384509 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.384462 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mkdkv"] May 28 03:17:16.384670 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.384587 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:16.384742 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:16.384681 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:17:16.435929 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.435846 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r4lqr"] May 28 03:17:16.436083 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:16.435986 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:16.436149 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:16.436093 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:17:17.875111 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:17.875071 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:17.875540 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:17.875161 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:17.875540 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:17.875196 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:17.875540 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:17.875202 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:17:17.875540 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:17.875288 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:17:17.875540 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:17.875389 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:17:18.017980 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:18.017944 2555 generic.go:358] "Generic (PLEG): container finished" podID="d3e7c7f4-9f55-4d7e-9f01-7025793968a6" containerID="f26bfd6e2cbda0765239c0c7ac07cc92d9ad329a14973061412c227e6ec99caf" exitCode=0 May 28 03:17:18.018144 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:18.018030 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4hk25" event={"ID":"d3e7c7f4-9f55-4d7e-9f01-7025793968a6","Type":"ContainerDied","Data":"f26bfd6e2cbda0765239c0c7ac07cc92d9ad329a14973061412c227e6ec99caf"} May 28 03:17:19.874864 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:19.874667 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:19.875255 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:19.874723 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:19.875255 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:19.874949 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r4lqr" podUID="0bfd38fd-ac06-4fc8-99e7-45f520c80876" May 28 03:17:19.875255 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:19.874743 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:19.875255 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:19.875014 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pkkqh" podUID="06e44704-03d7-4840-a298-191be4e1b255" May 28 03:17:19.875255 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:19.875113 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:17:20.022925 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:20.022892 2555 generic.go:358] "Generic (PLEG): container finished" podID="d3e7c7f4-9f55-4d7e-9f01-7025793968a6" containerID="350574a2b24e35a63cf06f13d5f1bf37b589010d71eaa423aa419e27f36be5d8" exitCode=0 May 28 03:17:20.023083 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:20.022943 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4hk25" event={"ID":"d3e7c7f4-9f55-4d7e-9f01-7025793968a6","Type":"ContainerDied","Data":"350574a2b24e35a63cf06f13d5f1bf37b589010d71eaa423aa419e27f36be5d8"} May 28 03:17:21.613689 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.613618 2555 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-36.ec2.internal" event="NodeReady" May 28 03:17:21.614094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.613738 2555 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 28 03:17:21.615928 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.615902 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:21.616049 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.615959 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:21.616096 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:21.616045 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 28 03:17:21.616096 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:21.616052 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:17:21.616157 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:21.616105 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret podName:06e44704-03d7-4840-a298-191be4e1b255 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:53.616087964 +0000 UTC m=+66.326733910 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret") pod "global-pull-secret-syncer-pkkqh" (UID: "06e44704-03d7-4840-a298-191be4e1b255") : object "kube-system"/"original-pull-secret" not registered May 28 03:17:21.616157 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:21.616120 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs podName:f8725dbf-86ef-41e7-845a-5b81d2343b2b nodeName:}" failed. No retries permitted until 2026-05-28 03:17:53.616114234 +0000 UTC m=+66.326760173 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs") pod "network-metrics-daemon-mkdkv" (UID: "f8725dbf-86ef-41e7-845a-5b81d2343b2b") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 03:17:21.691147 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.690720 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6fdb7dd57c-lm4nk"] May 28 03:17:21.705578 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.705542 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n"] May 28 03:17:21.705741 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.705701 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.716601 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.716576 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5mphw\" (UniqueName: \"kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw\") pod \"network-check-target-r4lqr\" (UID: \"0bfd38fd-ac06-4fc8-99e7-45f520c80876\") " pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:21.716771 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:21.716744 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 03:17:21.716771 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:21.716768 2555 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 03:17:21.716902 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:21.716781 2555 projected.go:194] Error preparing data for projected volume kube-api-access-5mphw for pod openshift-network-diagnostics/network-check-target-r4lqr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:17:21.716902 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:21.716851 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw podName:0bfd38fd-ac06-4fc8-99e7-45f520c80876 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:53.716832687 +0000 UTC m=+66.427478651 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5mphw" (UniqueName: "kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw") pod "network-check-target-r4lqr" (UID: "0bfd38fd-ac06-4fc8-99e7-45f520c80876") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 03:17:21.719827 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.719806 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl"] May 28 03:17:21.719958 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.719939 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.722792 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.722774 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" May 28 03:17:21.723175 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.723158 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" May 28 03:17:21.724630 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.724610 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" May 28 03:17:21.724721 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.724640 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4cm2m\"" May 28 03:17:21.732200 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.732175 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg"] May 28 03:17:21.732336 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.732319 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:17:21.734697 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.734679 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" May 28 03:17:21.735155 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.735127 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" May 28 03:17:21.735155 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.735148 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" May 28 03:17:21.735469 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.735448 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" May 28 03:17:21.736296 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.736276 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" May 28 03:17:21.736429 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.736409 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" May 28 03:17:21.736483 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.736415 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" May 28 03:17:21.737944 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.737927 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" May 28 03:17:21.742640 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.742622 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" May 28 03:17:21.748525 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.748506 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7"] May 28 03:17:21.748656 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.748638 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:17:21.765108 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.765085 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n"] May 28 03:17:21.765234 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.765212 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7" May 28 03:17:21.777170 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.777144 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fdb7dd57c-lm4nk"] May 28 03:17:21.778251 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.778232 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" May 28 03:17:21.778538 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.778521 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl"] May 28 03:17:21.779009 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.778986 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" May 28 03:17:21.779091 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.779044 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hb48f\"" May 28 03:17:21.779159 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.779146 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg"] May 28 03:17:21.779258 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.779244 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" May 28 03:17:21.779347 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.779306 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-22499\"" May 28 03:17:21.790872 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.790785 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7"] May 28 03:17:21.817417 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817382 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5cc4ec5c-3375-4364-96b0-6b8546bcf156-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.817612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817424 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/5cc4ec5c-3375-4364-96b0-6b8546bcf156-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.817612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817457 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5a87c119-6651-4c45-87be-302887a66bff-ca-trust-extracted\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.817612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817521 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.817612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817542 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/784177b5-9fdf-456f-a89a-a6e472a6ee32-tmp\") pod \"klusterlet-addon-workmgr-59bb78bb77-hz2rl\" (UID: \"784177b5-9fdf-456f-a89a-a6e472a6ee32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:17:21.817612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817564 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5a87c119-6651-4c45-87be-302887a66bff-image-registry-private-configuration\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.817612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817582 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5a87c119-6651-4c45-87be-302887a66bff-installation-pull-secrets\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.817823 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817673 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5cc4ec5c-3375-4364-96b0-6b8546bcf156-ca\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.817823 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817704 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a87c119-6651-4c45-87be-302887a66bff-trusted-ca\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.817823 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817727 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5a87c119-6651-4c45-87be-302887a66bff-registry-certificates\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.817823 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817755 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-bound-sa-token\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.817823 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817777 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5cc4ec5c-3375-4364-96b0-6b8546bcf156-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.817823 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817796 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/784177b5-9fdf-456f-a89a-a6e472a6ee32-klusterlet-config\") pod \"klusterlet-addon-workmgr-59bb78bb77-hz2rl\" (UID: \"784177b5-9fdf-456f-a89a-a6e472a6ee32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:17:21.817823 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817818 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jf85\" (UniqueName: \"kubernetes.io/projected/5cc4ec5c-3375-4364-96b0-6b8546bcf156-kube-api-access-2jf85\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.818036 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817843 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w4vd\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-kube-api-access-5w4vd\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.818036 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817863 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmtct\" (UniqueName: \"kubernetes.io/projected/784177b5-9fdf-456f-a89a-a6e472a6ee32-kube-api-access-fmtct\") pod \"klusterlet-addon-workmgr-59bb78bb77-hz2rl\" (UID: \"784177b5-9fdf-456f-a89a-a6e472a6ee32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:17:21.818036 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.817879 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5cc4ec5c-3375-4364-96b0-6b8546bcf156-hub\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.831208 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.831169 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kcl9f"] May 28 03:17:21.841841 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.841815 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8sbsk"] May 28 03:17:21.841976 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.841960 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kcl9f" May 28 03:17:21.844796 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.844778 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 28 03:17:21.845024 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.845008 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 28 03:17:21.845110 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.845026 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-krlmb\"" May 28 03:17:21.853886 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.853864 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:17:21.861780 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.861755 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 28 03:17:21.861896 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.861783 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 28 03:17:21.877577 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.874949 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:21.877577 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.874985 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:21.877577 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.875598 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:21.885562 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.885543 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 28 03:17:21.885664 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.885543 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 28 03:17:21.889772 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.889752 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 28 03:17:21.891225 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.891204 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 28 03:17:21.891382 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.891365 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pw92t\"" May 28 03:17:21.891617 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.891601 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vzb4w\"" May 28 03:17:21.891873 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.891856 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 28 03:17:21.891968 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.891953 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vsmkp\"" May 28 03:17:21.892202 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.892185 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kcl9f"] May 28 03:17:21.902607 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.902585 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8sbsk"] May 28 03:17:21.918924 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.918894 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5cc4ec5c-3375-4364-96b0-6b8546bcf156-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.919043 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.918929 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/5cc4ec5c-3375-4364-96b0-6b8546bcf156-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.919043 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.918952 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7497ac51-1873-4881-9f5d-fd40ca01efd9-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:17:21.919043 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.918984 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5a87c119-6651-4c45-87be-302887a66bff-ca-trust-extracted\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.919043 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919020 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.919308 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919045 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec456511-caf8-4385-909b-7c7ba33e1b1f-config-volume\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:21.919308 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919092 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/784177b5-9fdf-456f-a89a-a6e472a6ee32-tmp\") pod \"klusterlet-addon-workmgr-59bb78bb77-hz2rl\" (UID: \"784177b5-9fdf-456f-a89a-a6e472a6ee32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:17:21.919308 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919120 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5a87c119-6651-4c45-87be-302887a66bff-image-registry-private-configuration\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.919308 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:21.919126 2555 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 03:17:21.919308 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:21.919143 2555 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fdb7dd57c-lm4nk: secret "image-registry-tls" not found May 28 03:17:21.919308 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919145 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5a87c119-6651-4c45-87be-302887a66bff-installation-pull-secrets\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.919308 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919171 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9f6k\" (UniqueName: \"kubernetes.io/projected/ec456511-caf8-4385-909b-7c7ba33e1b1f-kube-api-access-f9f6k\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:21.919308 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:21.919212 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls podName:5a87c119-6651-4c45-87be-302887a66bff nodeName:}" failed. No retries permitted until 2026-05-28 03:17:22.419193096 +0000 UTC m=+35.129839054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls") pod "image-registry-6fdb7dd57c-lm4nk" (UID: "5a87c119-6651-4c45-87be-302887a66bff") : secret "image-registry-tls" not found May 28 03:17:21.919308 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919239 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:17:21.919308 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919272 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec456511-caf8-4385-909b-7c7ba33e1b1f-tmp-dir\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:21.919308 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919300 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg2n7\" (UniqueName: \"kubernetes.io/projected/0fd8b544-b597-48e3-9b6a-a4631e29100c-kube-api-access-cg2n7\") pod \"managed-serviceaccount-addon-agent-54b988fcdd-5nwk7\" (UID: \"0fd8b544-b597-48e3-9b6a-a4631e29100c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7" May 28 03:17:21.919778 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919352 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5cc4ec5c-3375-4364-96b0-6b8546bcf156-ca\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.919778 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919378 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a87c119-6651-4c45-87be-302887a66bff-trusted-ca\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.919778 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919404 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5a87c119-6651-4c45-87be-302887a66bff-registry-certificates\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.919778 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919432 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-bound-sa-token\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.919778 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919449 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/784177b5-9fdf-456f-a89a-a6e472a6ee32-tmp\") pod \"klusterlet-addon-workmgr-59bb78bb77-hz2rl\" (UID: \"784177b5-9fdf-456f-a89a-a6e472a6ee32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:17:21.919778 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919466 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5cc4ec5c-3375-4364-96b0-6b8546bcf156-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.919778 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919521 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/784177b5-9fdf-456f-a89a-a6e472a6ee32-klusterlet-config\") pod \"klusterlet-addon-workmgr-59bb78bb77-hz2rl\" (UID: \"784177b5-9fdf-456f-a89a-a6e472a6ee32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:17:21.919778 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919545 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2jf85\" (UniqueName: \"kubernetes.io/projected/5cc4ec5c-3375-4364-96b0-6b8546bcf156-kube-api-access-2jf85\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.919778 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919576 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5w4vd\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-kube-api-access-5w4vd\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.919778 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919603 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:21.919778 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919634 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fmtct\" (UniqueName: \"kubernetes.io/projected/784177b5-9fdf-456f-a89a-a6e472a6ee32-kube-api-access-fmtct\") pod \"klusterlet-addon-workmgr-59bb78bb77-hz2rl\" (UID: \"784177b5-9fdf-456f-a89a-a6e472a6ee32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:17:21.919778 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919659 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5cc4ec5c-3375-4364-96b0-6b8546bcf156-hub\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.919778 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919683 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fd8b544-b597-48e3-9b6a-a4631e29100c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54b988fcdd-5nwk7\" (UID: \"0fd8b544-b597-48e3-9b6a-a4631e29100c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7" May 28 03:17:21.920411 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919794 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/5cc4ec5c-3375-4364-96b0-6b8546bcf156-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.920411 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.919997 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5a87c119-6651-4c45-87be-302887a66bff-registry-certificates\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.920833 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.920775 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a87c119-6651-4c45-87be-302887a66bff-trusted-ca\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.923798 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.923645 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5a87c119-6651-4c45-87be-302887a66bff-ca-trust-extracted\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.924352 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.924052 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5a87c119-6651-4c45-87be-302887a66bff-installation-pull-secrets\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.924352 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.924070 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5cc4ec5c-3375-4364-96b0-6b8546bcf156-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.924352 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.924223 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5cc4ec5c-3375-4364-96b0-6b8546bcf156-hub\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.924352 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.924246 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/784177b5-9fdf-456f-a89a-a6e472a6ee32-klusterlet-config\") pod \"klusterlet-addon-workmgr-59bb78bb77-hz2rl\" (UID: \"784177b5-9fdf-456f-a89a-a6e472a6ee32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:17:21.924352 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.924268 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5cc4ec5c-3375-4364-96b0-6b8546bcf156-ca\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.924352 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.924288 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5cc4ec5c-3375-4364-96b0-6b8546bcf156-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.924352 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.924268 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5a87c119-6651-4c45-87be-302887a66bff-image-registry-private-configuration\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.944884 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.944850 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-bound-sa-token\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:21.944884 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.944877 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jf85\" (UniqueName: \"kubernetes.io/projected/5cc4ec5c-3375-4364-96b0-6b8546bcf156-kube-api-access-2jf85\") pod \"cluster-proxy-proxy-agent-6f7c789ff4-2ps4n\" (UID: \"5cc4ec5c-3375-4364-96b0-6b8546bcf156\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:21.947132 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.947111 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmtct\" (UniqueName: \"kubernetes.io/projected/784177b5-9fdf-456f-a89a-a6e472a6ee32-kube-api-access-fmtct\") pod \"klusterlet-addon-workmgr-59bb78bb77-hz2rl\" (UID: \"784177b5-9fdf-456f-a89a-a6e472a6ee32\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:17:21.952447 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:21.952422 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w4vd\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-kube-api-access-5w4vd\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:22.020547 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.020512 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:17:22.020547 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.020551 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cg2n7\" (UniqueName: \"kubernetes.io/projected/0fd8b544-b597-48e3-9b6a-a4631e29100c-kube-api-access-cg2n7\") pod \"managed-serviceaccount-addon-agent-54b988fcdd-5nwk7\" (UID: \"0fd8b544-b597-48e3-9b6a-a4631e29100c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7" May 28 03:17:22.020787 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.020594 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:22.020787 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.020619 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fd8b544-b597-48e3-9b6a-a4631e29100c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54b988fcdd-5nwk7\" (UID: \"0fd8b544-b597-48e3-9b6a-a4631e29100c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7" May 28 03:17:22.020787 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.020647 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 28 03:17:22.020787 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.020712 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert podName:7497ac51-1873-4881-9f5d-fd40ca01efd9 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:22.520697507 +0000 UTC m=+35.231343446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-pv6vg" (UID: "7497ac51-1873-4881-9f5d-fd40ca01efd9") : secret "networking-console-plugin-cert" not found May 28 03:17:22.020787 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.020730 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 03:17:22.020787 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.020654 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7497ac51-1873-4881-9f5d-fd40ca01efd9-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:17:22.021094 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.020794 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls podName:ec456511-caf8-4385-909b-7c7ba33e1b1f nodeName:}" failed. No retries permitted until 2026-05-28 03:17:22.520772154 +0000 UTC m=+35.231418102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls") pod "dns-default-kcl9f" (UID: "ec456511-caf8-4385-909b-7c7ba33e1b1f") : secret "dns-default-metrics-tls" not found May 28 03:17:22.021094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.020823 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec456511-caf8-4385-909b-7c7ba33e1b1f-tmp-dir\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:22.021094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.020859 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:17:22.021094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.020903 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec456511-caf8-4385-909b-7c7ba33e1b1f-config-volume\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:22.021094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.020947 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f9f6k\" (UniqueName: \"kubernetes.io/projected/ec456511-caf8-4385-909b-7c7ba33e1b1f-kube-api-access-f9f6k\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:22.021094 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.020965 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tnjw\" (UniqueName: \"kubernetes.io/projected/e6747510-251a-41cb-adef-fc02441f4e00-kube-api-access-9tnjw\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:17:22.021322 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.021113 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec456511-caf8-4385-909b-7c7ba33e1b1f-tmp-dir\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:22.021423 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.021405 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7497ac51-1873-4881-9f5d-fd40ca01efd9-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:17:22.021521 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.021481 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec456511-caf8-4385-909b-7c7ba33e1b1f-config-volume\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:22.023125 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.023102 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fd8b544-b597-48e3-9b6a-a4631e29100c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54b988fcdd-5nwk7\" (UID: \"0fd8b544-b597-48e3-9b6a-a4631e29100c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7" May 28 03:17:22.028959 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.028934 2555 generic.go:358] "Generic (PLEG): container finished" podID="d3e7c7f4-9f55-4d7e-9f01-7025793968a6" containerID="4df6ea838188183f5ac909bc2d138510ee281f8e3f4b5e788bc71363c914f98c" exitCode=0 May 28 03:17:22.029057 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.028971 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4hk25" event={"ID":"d3e7c7f4-9f55-4d7e-9f01-7025793968a6","Type":"ContainerDied","Data":"4df6ea838188183f5ac909bc2d138510ee281f8e3f4b5e788bc71363c914f98c"} May 28 03:17:22.036871 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.036850 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:17:22.044399 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.044380 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:17:22.051246 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.051226 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9f6k\" (UniqueName: \"kubernetes.io/projected/ec456511-caf8-4385-909b-7c7ba33e1b1f-kube-api-access-f9f6k\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:22.058846 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.058537 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg2n7\" (UniqueName: \"kubernetes.io/projected/0fd8b544-b597-48e3-9b6a-a4631e29100c-kube-api-access-cg2n7\") pod \"managed-serviceaccount-addon-agent-54b988fcdd-5nwk7\" (UID: \"0fd8b544-b597-48e3-9b6a-a4631e29100c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7" May 28 03:17:22.074477 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.074442 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7" May 28 03:17:22.126064 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.122324 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9tnjw\" (UniqueName: \"kubernetes.io/projected/e6747510-251a-41cb-adef-fc02441f4e00-kube-api-access-9tnjw\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:17:22.126064 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.122527 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:17:22.126064 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.124103 2555 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 03:17:22.126064 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.124175 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert podName:e6747510-251a-41cb-adef-fc02441f4e00 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:22.624156022 +0000 UTC m=+35.334801976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert") pod "ingress-canary-8sbsk" (UID: "e6747510-251a-41cb-adef-fc02441f4e00") : secret "canary-serving-cert" not found May 28 03:17:22.151482 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.151054 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tnjw\" (UniqueName: \"kubernetes.io/projected/e6747510-251a-41cb-adef-fc02441f4e00-kube-api-access-9tnjw\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:17:22.295635 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.295585 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7"] May 28 03:17:22.301303 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:17:22.301275 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fd8b544_b597_48e3_9b6a_a4631e29100c.slice/crio-10d29ca1f04fe9f8380ba0231a58ad697dc4559f8d1aae72e1191bb17f18df05 WatchSource:0}: Error finding container 10d29ca1f04fe9f8380ba0231a58ad697dc4559f8d1aae72e1191bb17f18df05: Status 404 returned error can't find the container with id 10d29ca1f04fe9f8380ba0231a58ad697dc4559f8d1aae72e1191bb17f18df05 May 28 03:17:22.315334 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:17:22.315304 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod784177b5_9fdf_456f_a89a_a6e472a6ee32.slice/crio-ece6b450dbfac675387459dea68ffc81b30f07c059aec341fba35ccf89eef426 WatchSource:0}: Error finding container ece6b450dbfac675387459dea68ffc81b30f07c059aec341fba35ccf89eef426: Status 404 returned error can't find the container with id ece6b450dbfac675387459dea68ffc81b30f07c059aec341fba35ccf89eef426 May 28 03:17:22.317817 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.317720 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl"] May 28 03:17:22.320947 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.320922 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n"] May 28 03:17:22.323818 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:17:22.323792 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5cc4ec5c_3375_4364_96b0_6b8546bcf156.slice/crio-73488584a9b551903186c940466406ef46ec692db2075301f6cc3e5c4954737d WatchSource:0}: Error finding container 73488584a9b551903186c940466406ef46ec692db2075301f6cc3e5c4954737d: Status 404 returned error can't find the container with id 73488584a9b551903186c940466406ef46ec692db2075301f6cc3e5c4954737d May 28 03:17:22.425144 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.425112 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:22.425311 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.425267 2555 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 03:17:22.425311 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.425288 2555 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fdb7dd57c-lm4nk: secret "image-registry-tls" not found May 28 03:17:22.425383 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.425362 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls podName:5a87c119-6651-4c45-87be-302887a66bff nodeName:}" failed. No retries permitted until 2026-05-28 03:17:23.425337758 +0000 UTC m=+36.135983699 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls") pod "image-registry-6fdb7dd57c-lm4nk" (UID: "5a87c119-6651-4c45-87be-302887a66bff") : secret "image-registry-tls" not found May 28 03:17:22.525803 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.525767 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:17:22.526003 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.525819 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:22.526003 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.525923 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 28 03:17:22.526003 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.525992 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert podName:7497ac51-1873-4881-9f5d-fd40ca01efd9 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:23.525975871 +0000 UTC m=+36.236621811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-pv6vg" (UID: "7497ac51-1873-4881-9f5d-fd40ca01efd9") : secret "networking-console-plugin-cert" not found May 28 03:17:22.526003 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.525932 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 03:17:22.526143 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.526025 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls podName:ec456511-caf8-4385-909b-7c7ba33e1b1f nodeName:}" failed. No retries permitted until 2026-05-28 03:17:23.526019054 +0000 UTC m=+36.236664994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls") pod "dns-default-kcl9f" (UID: "ec456511-caf8-4385-909b-7c7ba33e1b1f") : secret "dns-default-metrics-tls" not found May 28 03:17:22.626604 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:22.626568 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:17:22.627243 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.626715 2555 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 03:17:22.627243 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:22.626781 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert podName:e6747510-251a-41cb-adef-fc02441f4e00 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:23.626764975 +0000 UTC m=+36.337410916 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert") pod "ingress-canary-8sbsk" (UID: "e6747510-251a-41cb-adef-fc02441f4e00") : secret "canary-serving-cert" not found May 28 03:17:23.031414 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:23.031365 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" event={"ID":"5cc4ec5c-3375-4364-96b0-6b8546bcf156","Type":"ContainerStarted","Data":"73488584a9b551903186c940466406ef46ec692db2075301f6cc3e5c4954737d"} May 28 03:17:23.032266 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:23.032243 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" event={"ID":"784177b5-9fdf-456f-a89a-a6e472a6ee32","Type":"ContainerStarted","Data":"ece6b450dbfac675387459dea68ffc81b30f07c059aec341fba35ccf89eef426"} May 28 03:17:23.033021 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:23.032996 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7" event={"ID":"0fd8b544-b597-48e3-9b6a-a4631e29100c","Type":"ContainerStarted","Data":"10d29ca1f04fe9f8380ba0231a58ad697dc4559f8d1aae72e1191bb17f18df05"} May 28 03:17:23.432341 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:23.432299 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:23.432621 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:23.432448 2555 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 03:17:23.432621 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:23.432472 2555 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fdb7dd57c-lm4nk: secret "image-registry-tls" not found May 28 03:17:23.432621 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:23.432568 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls podName:5a87c119-6651-4c45-87be-302887a66bff nodeName:}" failed. No retries permitted until 2026-05-28 03:17:25.432545991 +0000 UTC m=+38.143191934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls") pod "image-registry-6fdb7dd57c-lm4nk" (UID: "5a87c119-6651-4c45-87be-302887a66bff") : secret "image-registry-tls" not found May 28 03:17:23.533853 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:23.533755 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:17:23.533853 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:23.533846 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:23.534107 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:23.534077 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 03:17:23.534221 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:23.534141 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls podName:ec456511-caf8-4385-909b-7c7ba33e1b1f nodeName:}" failed. No retries permitted until 2026-05-28 03:17:25.534122735 +0000 UTC m=+38.244768681 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls") pod "dns-default-kcl9f" (UID: "ec456511-caf8-4385-909b-7c7ba33e1b1f") : secret "dns-default-metrics-tls" not found May 28 03:17:23.534576 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:23.534556 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 28 03:17:23.534679 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:23.534613 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert podName:7497ac51-1873-4881-9f5d-fd40ca01efd9 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:25.534597491 +0000 UTC m=+38.245243437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-pv6vg" (UID: "7497ac51-1873-4881-9f5d-fd40ca01efd9") : secret "networking-console-plugin-cert" not found May 28 03:17:23.634885 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:23.634847 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:17:23.635402 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:23.635092 2555 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 03:17:23.635402 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:23.635304 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert podName:e6747510-251a-41cb-adef-fc02441f4e00 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:25.63528206 +0000 UTC m=+38.345928023 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert") pod "ingress-canary-8sbsk" (UID: "e6747510-251a-41cb-adef-fc02441f4e00") : secret "canary-serving-cert" not found May 28 03:17:25.454316 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:25.454271 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:25.454759 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:25.454444 2555 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 03:17:25.454759 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:25.454470 2555 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fdb7dd57c-lm4nk: secret "image-registry-tls" not found May 28 03:17:25.454759 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:25.454555 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls podName:5a87c119-6651-4c45-87be-302887a66bff nodeName:}" failed. No retries permitted until 2026-05-28 03:17:29.45453288 +0000 UTC m=+42.165178825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls") pod "image-registry-6fdb7dd57c-lm4nk" (UID: "5a87c119-6651-4c45-87be-302887a66bff") : secret "image-registry-tls" not found May 28 03:17:25.555159 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:25.555111 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:17:25.555353 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:25.555189 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:25.555353 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:25.555281 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 28 03:17:25.555353 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:25.555313 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 03:17:25.555513 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:25.555372 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert podName:7497ac51-1873-4881-9f5d-fd40ca01efd9 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:29.555349684 +0000 UTC m=+42.265995626 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-pv6vg" (UID: "7497ac51-1873-4881-9f5d-fd40ca01efd9") : secret "networking-console-plugin-cert" not found May 28 03:17:25.555513 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:25.555395 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls podName:ec456511-caf8-4385-909b-7c7ba33e1b1f nodeName:}" failed. No retries permitted until 2026-05-28 03:17:29.5553855 +0000 UTC m=+42.266031445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls") pod "dns-default-kcl9f" (UID: "ec456511-caf8-4385-909b-7c7ba33e1b1f") : secret "dns-default-metrics-tls" not found May 28 03:17:25.656626 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:25.656581 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:17:25.656879 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:25.656860 2555 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 03:17:25.656962 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:25.656929 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert podName:e6747510-251a-41cb-adef-fc02441f4e00 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:29.656911166 +0000 UTC m=+42.367557121 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert") pod "ingress-canary-8sbsk" (UID: "e6747510-251a-41cb-adef-fc02441f4e00") : secret "canary-serving-cert" not found May 28 03:17:29.489729 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:29.489681 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:29.490230 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:29.489801 2555 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 03:17:29.490230 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:29.489829 2555 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fdb7dd57c-lm4nk: secret "image-registry-tls" not found May 28 03:17:29.490230 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:29.489904 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls podName:5a87c119-6651-4c45-87be-302887a66bff nodeName:}" failed. No retries permitted until 2026-05-28 03:17:37.489880648 +0000 UTC m=+50.200526591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls") pod "image-registry-6fdb7dd57c-lm4nk" (UID: "5a87c119-6651-4c45-87be-302887a66bff") : secret "image-registry-tls" not found May 28 03:17:29.590966 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:29.590909 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:17:29.591151 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:29.590985 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:29.591151 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:29.591089 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 28 03:17:29.591151 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:29.591116 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 03:17:29.591281 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:29.591172 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert podName:7497ac51-1873-4881-9f5d-fd40ca01efd9 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:37.591149863 +0000 UTC m=+50.301795831 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-pv6vg" (UID: "7497ac51-1873-4881-9f5d-fd40ca01efd9") : secret "networking-console-plugin-cert" not found May 28 03:17:29.591281 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:29.591194 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls podName:ec456511-caf8-4385-909b-7c7ba33e1b1f nodeName:}" failed. No retries permitted until 2026-05-28 03:17:37.591183975 +0000 UTC m=+50.301829929 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls") pod "dns-default-kcl9f" (UID: "ec456511-caf8-4385-909b-7c7ba33e1b1f") : secret "dns-default-metrics-tls" not found May 28 03:17:29.692461 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:29.692424 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:17:29.692662 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:29.692589 2555 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 03:17:29.692662 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:29.692653 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert podName:e6747510-251a-41cb-adef-fc02441f4e00 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:37.692637149 +0000 UTC m=+50.403283089 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert") pod "ingress-canary-8sbsk" (UID: "e6747510-251a-41cb-adef-fc02441f4e00") : secret "canary-serving-cert" not found May 28 03:17:33.056669 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:33.056633 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7" event={"ID":"0fd8b544-b597-48e3-9b6a-a4631e29100c","Type":"ContainerStarted","Data":"2cebbeb18fa52e7bacecb964907fb96731790b3d93bed2993054aa0d296f6ec7"} May 28 03:17:33.058039 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:33.058013 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" event={"ID":"5cc4ec5c-3375-4364-96b0-6b8546bcf156","Type":"ContainerStarted","Data":"8764fa1cd063da2467604f86ff270af8df3e772ed63c3ce8a55b47c9b7b1257b"} May 28 03:17:33.060216 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:33.060194 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4hk25" event={"ID":"d3e7c7f4-9f55-4d7e-9f01-7025793968a6","Type":"ContainerStarted","Data":"b73147d0546077f01d552fe1bac7c746265daa704eb2479dc907f7eea39871c2"} May 28 03:17:33.088943 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:33.088892 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7" podStartSLOduration=10.670663072 podStartE2EDuration="21.088874296s" podCreationTimestamp="2026-05-28 03:17:12 +0000 UTC" firstStartedPulling="2026-05-28 03:17:22.302931897 +0000 UTC m=+35.013577836" lastFinishedPulling="2026-05-28 03:17:32.72114312 +0000 UTC m=+45.431789060" observedRunningTime="2026-05-28 03:17:33.088263981 +0000 UTC m=+45.798909943" watchObservedRunningTime="2026-05-28 03:17:33.088874296 +0000 UTC m=+45.799520258" May 28 03:17:34.064725 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:34.064691 2555 generic.go:358] "Generic (PLEG): container finished" podID="d3e7c7f4-9f55-4d7e-9f01-7025793968a6" containerID="b73147d0546077f01d552fe1bac7c746265daa704eb2479dc907f7eea39871c2" exitCode=0 May 28 03:17:34.065144 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:34.064778 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4hk25" event={"ID":"d3e7c7f4-9f55-4d7e-9f01-7025793968a6","Type":"ContainerDied","Data":"b73147d0546077f01d552fe1bac7c746265daa704eb2479dc907f7eea39871c2"} May 28 03:17:34.066052 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:34.066021 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" event={"ID":"784177b5-9fdf-456f-a89a-a6e472a6ee32","Type":"ContainerStarted","Data":"47716b11bfb8c4e6ccd82e6cab5ba83df9664655fda8829964751b0a6a428532"} May 28 03:17:34.129926 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:34.129868 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" podStartSLOduration=11.277345038 podStartE2EDuration="22.129848882s" podCreationTimestamp="2026-05-28 03:17:12 +0000 UTC" firstStartedPulling="2026-05-28 03:17:22.317437879 +0000 UTC m=+35.028083820" lastFinishedPulling="2026-05-28 03:17:33.169941715 +0000 UTC m=+45.880587664" observedRunningTime="2026-05-28 03:17:34.129102096 +0000 UTC m=+46.839748059" watchObservedRunningTime="2026-05-28 03:17:34.129848882 +0000 UTC m=+46.840494844" May 28 03:17:35.070470 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:35.070437 2555 generic.go:358] "Generic (PLEG): container finished" podID="d3e7c7f4-9f55-4d7e-9f01-7025793968a6" containerID="d12df3a0c4c395c7394b71b5711118f55455b6ebe6783c72cef2ba50a44c6a97" exitCode=0 May 28 03:17:35.070942 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:35.070522 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4hk25" event={"ID":"d3e7c7f4-9f55-4d7e-9f01-7025793968a6","Type":"ContainerDied","Data":"d12df3a0c4c395c7394b71b5711118f55455b6ebe6783c72cef2ba50a44c6a97"} May 28 03:17:35.070942 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:35.070881 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:17:35.072851 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:35.072828 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:17:36.076535 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:36.076478 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4hk25" event={"ID":"d3e7c7f4-9f55-4d7e-9f01-7025793968a6","Type":"ContainerStarted","Data":"30ba9a27dfc705bcbfed8fcc5c8dc7fb152e060ff186de345c02b77ff7d0c6fb"} May 28 03:17:36.124099 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:36.124039 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4hk25" podStartSLOduration=6.975229636 podStartE2EDuration="49.124013521s" podCreationTimestamp="2026-05-28 03:16:47 +0000 UTC" firstStartedPulling="2026-05-28 03:16:50.595966662 +0000 UTC m=+3.306612604" lastFinishedPulling="2026-05-28 03:17:32.744750547 +0000 UTC m=+45.455396489" observedRunningTime="2026-05-28 03:17:36.123474027 +0000 UTC m=+48.834119990" watchObservedRunningTime="2026-05-28 03:17:36.124013521 +0000 UTC m=+48.834659484" May 28 03:17:37.080404 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:37.080362 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" event={"ID":"5cc4ec5c-3375-4364-96b0-6b8546bcf156","Type":"ContainerStarted","Data":"ef5ba60be097d88f43ce3a7242a9fcbff017e0750148d8ec42aa822e3abd65cd"} May 28 03:17:37.080404 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:37.080402 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" event={"ID":"5cc4ec5c-3375-4364-96b0-6b8546bcf156","Type":"ContainerStarted","Data":"0dbdb83456087c0dfc79b3f8fd3afac651f91b11456094706e174344a1379d7b"} May 28 03:17:37.557786 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:37.557749 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:37.557974 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:37.557906 2555 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 03:17:37.557974 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:37.557927 2555 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fdb7dd57c-lm4nk: secret "image-registry-tls" not found May 28 03:17:37.558054 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:37.557994 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls podName:5a87c119-6651-4c45-87be-302887a66bff nodeName:}" failed. No retries permitted until 2026-05-28 03:17:53.557977542 +0000 UTC m=+66.268623482 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls") pod "image-registry-6fdb7dd57c-lm4nk" (UID: "5a87c119-6651-4c45-87be-302887a66bff") : secret "image-registry-tls" not found May 28 03:17:37.658320 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:37.658284 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:37.658460 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:37.658416 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 03:17:37.658541 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:37.658470 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls podName:ec456511-caf8-4385-909b-7c7ba33e1b1f nodeName:}" failed. No retries permitted until 2026-05-28 03:17:53.658455978 +0000 UTC m=+66.369101918 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls") pod "dns-default-kcl9f" (UID: "ec456511-caf8-4385-909b-7c7ba33e1b1f") : secret "dns-default-metrics-tls" not found May 28 03:17:37.658541 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:37.658416 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:17:37.658541 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:37.658519 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 28 03:17:37.658673 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:37.658579 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert podName:7497ac51-1873-4881-9f5d-fd40ca01efd9 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:53.658563135 +0000 UTC m=+66.369209075 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-pv6vg" (UID: "7497ac51-1873-4881-9f5d-fd40ca01efd9") : secret "networking-console-plugin-cert" not found May 28 03:17:37.759439 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:37.759401 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:17:37.759609 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:37.759558 2555 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 03:17:37.759648 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:37.759618 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert podName:e6747510-251a-41cb-adef-fc02441f4e00 nodeName:}" failed. No retries permitted until 2026-05-28 03:17:53.759601208 +0000 UTC m=+66.470247193 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert") pod "ingress-canary-8sbsk" (UID: "e6747510-251a-41cb-adef-fc02441f4e00") : secret "canary-serving-cert" not found May 28 03:17:48.028828 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:48.028797 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mqxrv" May 28 03:17:48.074383 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:48.074330 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" podStartSLOduration=22.356644214 podStartE2EDuration="36.074316785s" podCreationTimestamp="2026-05-28 03:17:12 +0000 UTC" firstStartedPulling="2026-05-28 03:17:22.325547207 +0000 UTC m=+35.036193147" lastFinishedPulling="2026-05-28 03:17:36.043219776 +0000 UTC m=+48.753865718" observedRunningTime="2026-05-28 03:17:37.112465833 +0000 UTC m=+49.823111795" watchObservedRunningTime="2026-05-28 03:17:48.074316785 +0000 UTC m=+60.784962746" May 28 03:17:53.586680 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.586634 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:17:53.587077 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:53.586787 2555 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 03:17:53.587077 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:53.586810 2555 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fdb7dd57c-lm4nk: secret "image-registry-tls" not found May 28 03:17:53.587077 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:53.586868 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls podName:5a87c119-6651-4c45-87be-302887a66bff nodeName:}" failed. No retries permitted until 2026-05-28 03:18:25.586853224 +0000 UTC m=+98.297499164 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls") pod "image-registry-6fdb7dd57c-lm4nk" (UID: "5a87c119-6651-4c45-87be-302887a66bff") : secret "image-registry-tls" not found May 28 03:17:53.687050 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.687007 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:17:53.687050 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.687055 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:53.687276 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.687074 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:17:53.687276 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.687130 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:17:53.687276 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:53.687149 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 28 03:17:53.687276 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:53.687213 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert podName:7497ac51-1873-4881-9f5d-fd40ca01efd9 nodeName:}" failed. No retries permitted until 2026-05-28 03:18:25.687198356 +0000 UTC m=+98.397844296 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-pv6vg" (UID: "7497ac51-1873-4881-9f5d-fd40ca01efd9") : secret "networking-console-plugin-cert" not found May 28 03:17:53.687276 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:53.687245 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 03:17:53.687480 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:53.687307 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls podName:ec456511-caf8-4385-909b-7c7ba33e1b1f nodeName:}" failed. No retries permitted until 2026-05-28 03:18:25.687286168 +0000 UTC m=+98.397932128 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls") pod "dns-default-kcl9f" (UID: "ec456511-caf8-4385-909b-7c7ba33e1b1f") : secret "dns-default-metrics-tls" not found May 28 03:17:53.690409 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.690384 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 28 03:17:53.690537 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.690514 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 28 03:17:53.698145 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:53.698120 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 28 03:17:53.698260 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:53.698184 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs podName:f8725dbf-86ef-41e7-845a-5b81d2343b2b nodeName:}" failed. No retries permitted until 2026-05-28 03:18:57.69816946 +0000 UTC m=+130.408815400 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs") pod "network-metrics-daemon-mkdkv" (UID: "f8725dbf-86ef-41e7-845a-5b81d2343b2b") : secret "metrics-daemon-secret" not found May 28 03:17:53.701214 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.701192 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/06e44704-03d7-4840-a298-191be4e1b255-original-pull-secret\") pod \"global-pull-secret-syncer-pkkqh\" (UID: \"06e44704-03d7-4840-a298-191be4e1b255\") " pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:53.787974 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.787938 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5mphw\" (UniqueName: \"kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw\") pod \"network-check-target-r4lqr\" (UID: \"0bfd38fd-ac06-4fc8-99e7-45f520c80876\") " pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:53.787974 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.787990 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:17:53.788203 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:53.788114 2555 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 03:17:53.788203 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:17:53.788167 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert podName:e6747510-251a-41cb-adef-fc02441f4e00 nodeName:}" failed. No retries permitted until 2026-05-28 03:18:25.788152834 +0000 UTC m=+98.498798773 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert") pod "ingress-canary-8sbsk" (UID: "e6747510-251a-41cb-adef-fc02441f4e00") : secret "canary-serving-cert" not found May 28 03:17:53.791223 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.791204 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 28 03:17:53.801946 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.801921 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 28 03:17:53.812136 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.812100 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mphw\" (UniqueName: \"kubernetes.io/projected/0bfd38fd-ac06-4fc8-99e7-45f520c80876-kube-api-access-5mphw\") pod \"network-check-target-r4lqr\" (UID: \"0bfd38fd-ac06-4fc8-99e7-45f520c80876\") " pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:53.986724 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:53.986693 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pkkqh" May 28 03:17:54.001605 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:54.001580 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pw92t\"" May 28 03:17:54.009414 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:54.009380 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:54.137300 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:54.137276 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pkkqh"] May 28 03:17:54.139920 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:17:54.139889 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e44704_03d7_4840_a298_191be4e1b255.slice/crio-28c31df2a0e7039b546749e8d77bcd310ecab3a3474add17db110c618b5a5656 WatchSource:0}: Error finding container 28c31df2a0e7039b546749e8d77bcd310ecab3a3474add17db110c618b5a5656: Status 404 returned error can't find the container with id 28c31df2a0e7039b546749e8d77bcd310ecab3a3474add17db110c618b5a5656 May 28 03:17:54.174242 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:54.174202 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r4lqr"] May 28 03:17:54.177848 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:17:54.177819 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bfd38fd_ac06_4fc8_99e7_45f520c80876.slice/crio-a7479d341be6674087d5d2ca9689335896382eb834aadda0397eca2c43bf90c0 WatchSource:0}: Error finding container a7479d341be6674087d5d2ca9689335896382eb834aadda0397eca2c43bf90c0: Status 404 returned error can't find the container with id a7479d341be6674087d5d2ca9689335896382eb834aadda0397eca2c43bf90c0 May 28 03:17:55.115668 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:55.115621 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r4lqr" event={"ID":"0bfd38fd-ac06-4fc8-99e7-45f520c80876","Type":"ContainerStarted","Data":"a7479d341be6674087d5d2ca9689335896382eb834aadda0397eca2c43bf90c0"} May 28 03:17:55.116959 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:55.116932 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pkkqh" event={"ID":"06e44704-03d7-4840-a298-191be4e1b255","Type":"ContainerStarted","Data":"28c31df2a0e7039b546749e8d77bcd310ecab3a3474add17db110c618b5a5656"} May 28 03:17:59.128109 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:59.127999 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pkkqh" event={"ID":"06e44704-03d7-4840-a298-191be4e1b255","Type":"ContainerStarted","Data":"d8aec5267378803c40d5df22f316de8d3c2dfff835f1c9831f9b1eaf91d10afa"} May 28 03:17:59.129423 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:59.129397 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r4lqr" event={"ID":"0bfd38fd-ac06-4fc8-99e7-45f520c80876","Type":"ContainerStarted","Data":"811a11da406576a135cf437ca6be39c3d3f36dad33966dd99a5955388d3466ba"} May 28 03:17:59.129571 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:59.129546 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:17:59.144533 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:59.144447 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pkkqh" podStartSLOduration=65.439256241 podStartE2EDuration="1m10.144427981s" podCreationTimestamp="2026-05-28 03:16:49 +0000 UTC" firstStartedPulling="2026-05-28 03:17:54.141825298 +0000 UTC m=+66.852471238" lastFinishedPulling="2026-05-28 03:17:58.846997035 +0000 UTC m=+71.557642978" observedRunningTime="2026-05-28 03:17:59.144166596 +0000 UTC m=+71.854812559" watchObservedRunningTime="2026-05-28 03:17:59.144427981 +0000 UTC m=+71.855073949" May 28 03:17:59.163178 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:17:59.163127 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r4lqr" podStartSLOduration=66.500776899 podStartE2EDuration="1m11.163110188s" podCreationTimestamp="2026-05-28 03:16:48 +0000 UTC" firstStartedPulling="2026-05-28 03:17:54.179697385 +0000 UTC m=+66.890343326" lastFinishedPulling="2026-05-28 03:17:58.842030661 +0000 UTC m=+71.552676615" observedRunningTime="2026-05-28 03:17:59.161081721 +0000 UTC m=+71.871727686" watchObservedRunningTime="2026-05-28 03:17:59.163110188 +0000 UTC m=+71.873756144" May 28 03:18:25.647950 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:18:25.647797 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:18:25.648449 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:18:25.647953 2555 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 03:18:25.648449 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:18:25.647973 2555 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fdb7dd57c-lm4nk: secret "image-registry-tls" not found May 28 03:18:25.648449 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:18:25.648042 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls podName:5a87c119-6651-4c45-87be-302887a66bff nodeName:}" failed. No retries permitted until 2026-05-28 03:19:29.648026295 +0000 UTC m=+162.358672235 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls") pod "image-registry-6fdb7dd57c-lm4nk" (UID: "5a87c119-6651-4c45-87be-302887a66bff") : secret "image-registry-tls" not found May 28 03:18:25.748824 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:18:25.748784 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:18:25.748947 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:18:25.748831 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:18:25.748982 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:18:25.748942 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 28 03:18:25.749025 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:18:25.749013 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert podName:7497ac51-1873-4881-9f5d-fd40ca01efd9 nodeName:}" failed. No retries permitted until 2026-05-28 03:19:29.748998805 +0000 UTC m=+162.459644750 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-pv6vg" (UID: "7497ac51-1873-4881-9f5d-fd40ca01efd9") : secret "networking-console-plugin-cert" not found May 28 03:18:25.749067 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:18:25.748942 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 03:18:25.749098 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:18:25.749072 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls podName:ec456511-caf8-4385-909b-7c7ba33e1b1f nodeName:}" failed. No retries permitted until 2026-05-28 03:19:29.749061782 +0000 UTC m=+162.459707721 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls") pod "dns-default-kcl9f" (UID: "ec456511-caf8-4385-909b-7c7ba33e1b1f") : secret "dns-default-metrics-tls" not found May 28 03:18:25.850079 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:18:25.850045 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:18:25.850226 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:18:25.850204 2555 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 03:18:25.850291 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:18:25.850281 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert podName:e6747510-251a-41cb-adef-fc02441f4e00 nodeName:}" failed. No retries permitted until 2026-05-28 03:19:29.850265593 +0000 UTC m=+162.560911538 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert") pod "ingress-canary-8sbsk" (UID: "e6747510-251a-41cb-adef-fc02441f4e00") : secret "canary-serving-cert" not found May 28 03:18:30.134519 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:18:30.134471 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r4lqr" May 28 03:18:57.795698 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:18:57.795656 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:18:57.796185 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:18:57.795801 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 28 03:18:57.796185 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:18:57.795880 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs podName:f8725dbf-86ef-41e7-845a-5b81d2343b2b nodeName:}" failed. No retries permitted until 2026-05-28 03:20:59.795862653 +0000 UTC m=+252.506508594 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs") pod "network-metrics-daemon-mkdkv" (UID: "f8725dbf-86ef-41e7-845a-5b81d2343b2b") : secret "metrics-daemon-secret" not found May 28 03:19:17.046064 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:17.046034 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gbftm_9d2867de-afec-49ff-be90-2680f09ddc2e/dns-node-resolver/0.log" May 28 03:19:18.049907 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:18.049878 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zx6tr_900e522d-e9a0-4344-ad62-c0518eb0cba7/node-ca/0.log" May 28 03:19:24.716328 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:19:24.716280 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" podUID="5a87c119-6651-4c45-87be-302887a66bff" May 28 03:19:24.756603 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:19:24.756559 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" podUID="7497ac51-1873-4881-9f5d-fd40ca01efd9" May 28 03:19:24.866006 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:19:24.865943 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-kcl9f" podUID="ec456511-caf8-4385-909b-7c7ba33e1b1f" May 28 03:19:24.871095 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:19:24.871059 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-8sbsk" podUID="e6747510-251a-41cb-adef-fc02441f4e00" May 28 03:19:24.893875 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:19:24.893828 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mkdkv" podUID="f8725dbf-86ef-41e7-845a-5b81d2343b2b" May 28 03:19:25.338710 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:25.338673 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:19:25.338710 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:25.338692 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:19:25.338710 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:25.338696 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kcl9f" May 28 03:19:25.338950 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:25.338674 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:19:29.657850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:29.657809 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:19:29.660178 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:29.660156 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls\") pod \"image-registry-6fdb7dd57c-lm4nk\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:19:29.759309 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:29.759253 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:19:29.759309 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:29.759313 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:19:29.759621 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:19:29.759423 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 28 03:19:29.759621 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:19:29.759543 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert podName:7497ac51-1873-4881-9f5d-fd40ca01efd9 nodeName:}" failed. No retries permitted until 2026-05-28 03:21:31.759521048 +0000 UTC m=+284.470166990 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-pv6vg" (UID: "7497ac51-1873-4881-9f5d-fd40ca01efd9") : secret "networking-console-plugin-cert" not found May 28 03:19:29.761666 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:29.761642 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec456511-caf8-4385-909b-7c7ba33e1b1f-metrics-tls\") pod \"dns-default-kcl9f\" (UID: \"ec456511-caf8-4385-909b-7c7ba33e1b1f\") " pod="openshift-dns/dns-default-kcl9f" May 28 03:19:29.843451 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:29.843421 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4cm2m\"" May 28 03:19:29.843694 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:29.843421 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-krlmb\"" May 28 03:19:29.850807 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:29.850784 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kcl9f" May 28 03:19:29.850887 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:29.850862 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:19:29.859603 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:29.859578 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:19:29.862226 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:29.862204 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6747510-251a-41cb-adef-fc02441f4e00-cert\") pod \"ingress-canary-8sbsk\" (UID: \"e6747510-251a-41cb-adef-fc02441f4e00\") " pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:19:29.986610 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:29.986579 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kcl9f"] May 28 03:19:29.989742 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:19:29.989715 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec456511_caf8_4385_909b_7c7ba33e1b1f.slice/crio-e74252a564b537d1f91b24b4e22727f279221e37dea3e3c454db744e13b0016d WatchSource:0}: Error finding container e74252a564b537d1f91b24b4e22727f279221e37dea3e3c454db744e13b0016d: Status 404 returned error can't find the container with id e74252a564b537d1f91b24b4e22727f279221e37dea3e3c454db744e13b0016d May 28 03:19:30.005658 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:30.005623 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fdb7dd57c-lm4nk"] May 28 03:19:30.008946 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:19:30.008919 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a87c119_6651_4c45_87be_302887a66bff.slice/crio-bdbc0189b653fd2f3c177331c7f95569c288895da75172cb64e44ae7d8243b2a WatchSource:0}: Error finding container bdbc0189b653fd2f3c177331c7f95569c288895da75172cb64e44ae7d8243b2a: Status 404 returned error can't find the container with id bdbc0189b653fd2f3c177331c7f95569c288895da75172cb64e44ae7d8243b2a May 28 03:19:30.141885 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:30.141858 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vsmkp\"" May 28 03:19:30.149928 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:30.149911 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8sbsk" May 28 03:19:30.275718 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:30.275688 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8sbsk"] May 28 03:19:30.278780 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:19:30.278755 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6747510_251a_41cb_adef_fc02441f4e00.slice/crio-d2b34034b20863b65e356487a7cc7e2525211e27952015331b77d4ec49188a56 WatchSource:0}: Error finding container d2b34034b20863b65e356487a7cc7e2525211e27952015331b77d4ec49188a56: Status 404 returned error can't find the container with id d2b34034b20863b65e356487a7cc7e2525211e27952015331b77d4ec49188a56 May 28 03:19:30.353016 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:30.352983 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8sbsk" event={"ID":"e6747510-251a-41cb-adef-fc02441f4e00","Type":"ContainerStarted","Data":"d2b34034b20863b65e356487a7cc7e2525211e27952015331b77d4ec49188a56"} May 28 03:19:30.354302 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:30.354275 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kcl9f" event={"ID":"ec456511-caf8-4385-909b-7c7ba33e1b1f","Type":"ContainerStarted","Data":"e74252a564b537d1f91b24b4e22727f279221e37dea3e3c454db744e13b0016d"} May 28 03:19:30.355616 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:30.355574 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" event={"ID":"5a87c119-6651-4c45-87be-302887a66bff","Type":"ContainerStarted","Data":"11804e82aa701954d2e09f66e52105578c2414605585f846ff307dba094679e2"} May 28 03:19:30.355718 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:30.355628 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" event={"ID":"5a87c119-6651-4c45-87be-302887a66bff","Type":"ContainerStarted","Data":"bdbc0189b653fd2f3c177331c7f95569c288895da75172cb64e44ae7d8243b2a"} May 28 03:19:30.355718 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:30.355664 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:19:30.379920 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:30.379873 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" podStartSLOduration=161.37985702 podStartE2EDuration="2m41.37985702s" podCreationTimestamp="2026-05-28 03:16:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 03:19:30.379431685 +0000 UTC m=+163.090077647" watchObservedRunningTime="2026-05-28 03:19:30.37985702 +0000 UTC m=+163.090502982" May 28 03:19:32.363591 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:32.363554 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kcl9f" event={"ID":"ec456511-caf8-4385-909b-7c7ba33e1b1f","Type":"ContainerStarted","Data":"4d355c2facdfb9fea9f97653a41d288f16cf240f375e083dc5ab7be254c52106"} May 28 03:19:32.363954 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:32.363599 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kcl9f" event={"ID":"ec456511-caf8-4385-909b-7c7ba33e1b1f","Type":"ContainerStarted","Data":"063f983eff817f481a85c7adc80cb3c4be35a51daf551054afa26dc01d211a82"} May 28 03:19:32.363954 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:32.363715 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kcl9f" May 28 03:19:32.387852 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:32.387791 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-kcl9f" podStartSLOduration=129.642241282 podStartE2EDuration="2m11.387775579s" podCreationTimestamp="2026-05-28 03:17:21 +0000 UTC" firstStartedPulling="2026-05-28 03:19:29.991431965 +0000 UTC m=+162.702077906" lastFinishedPulling="2026-05-28 03:19:31.736966258 +0000 UTC m=+164.447612203" observedRunningTime="2026-05-28 03:19:32.387760529 +0000 UTC m=+165.098406492" watchObservedRunningTime="2026-05-28 03:19:32.387775579 +0000 UTC m=+165.098421538" May 28 03:19:33.367469 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:33.367425 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8sbsk" event={"ID":"e6747510-251a-41cb-adef-fc02441f4e00","Type":"ContainerStarted","Data":"5121b0fd1d12150278f457821dfc12295a2d24d93de9072a77132fc43148ce61"} May 28 03:19:33.368802 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:33.368776 2555 generic.go:358] "Generic (PLEG): container finished" podID="0fd8b544-b597-48e3-9b6a-a4631e29100c" containerID="2cebbeb18fa52e7bacecb964907fb96731790b3d93bed2993054aa0d296f6ec7" exitCode=255 May 28 03:19:33.368911 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:33.368855 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7" event={"ID":"0fd8b544-b597-48e3-9b6a-a4631e29100c","Type":"ContainerDied","Data":"2cebbeb18fa52e7bacecb964907fb96731790b3d93bed2993054aa0d296f6ec7"} May 28 03:19:33.369272 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:33.369256 2555 scope.go:117] "RemoveContainer" containerID="2cebbeb18fa52e7bacecb964907fb96731790b3d93bed2993054aa0d296f6ec7" May 28 03:19:33.382193 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:33.382157 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8sbsk" podStartSLOduration=130.241598796 podStartE2EDuration="2m12.382143345s" podCreationTimestamp="2026-05-28 03:17:21 +0000 UTC" firstStartedPulling="2026-05-28 03:19:30.280585008 +0000 UTC m=+162.991230948" lastFinishedPulling="2026-05-28 03:19:32.421129551 +0000 UTC m=+165.131775497" observedRunningTime="2026-05-28 03:19:33.381868021 +0000 UTC m=+166.092513995" watchObservedRunningTime="2026-05-28 03:19:33.382143345 +0000 UTC m=+166.092789301" May 28 03:19:34.372842 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:34.372810 2555 generic.go:358] "Generic (PLEG): container finished" podID="784177b5-9fdf-456f-a89a-a6e472a6ee32" containerID="47716b11bfb8c4e6ccd82e6cab5ba83df9664655fda8829964751b0a6a428532" exitCode=1 May 28 03:19:34.373251 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:34.372886 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" event={"ID":"784177b5-9fdf-456f-a89a-a6e472a6ee32","Type":"ContainerDied","Data":"47716b11bfb8c4e6ccd82e6cab5ba83df9664655fda8829964751b0a6a428532"} May 28 03:19:34.373251 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:34.373237 2555 scope.go:117] "RemoveContainer" containerID="47716b11bfb8c4e6ccd82e6cab5ba83df9664655fda8829964751b0a6a428532" May 28 03:19:34.374469 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:34.374381 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b988fcdd-5nwk7" event={"ID":"0fd8b544-b597-48e3-9b6a-a4631e29100c","Type":"ContainerStarted","Data":"9403a683238d7bcfe541c0487458ac441d52e286d737b286ff07d02c4fc0621e"} May 28 03:19:35.071354 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:35.071307 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:19:35.378873 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:35.378794 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" event={"ID":"784177b5-9fdf-456f-a89a-a6e472a6ee32","Type":"ContainerStarted","Data":"32b3cdbba8d91dac7c08cd9f972bcd939388fb28e17b78f64ec2de3ddc9898ea"} May 28 03:19:35.379236 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:35.378986 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:19:35.379677 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:35.379653 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-59bb78bb77-hz2rl" May 28 03:19:37.875103 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:37.875025 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:19:40.967045 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:40.967003 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-849pb"] May 28 03:19:40.970265 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:40.970241 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:40.973592 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:40.973571 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xkttp\"" May 28 03:19:40.973864 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:40.973842 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 28 03:19:40.974654 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:40.974638 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 28 03:19:40.974754 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:40.974662 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 28 03:19:40.974754 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:40.974641 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 28 03:19:41.032010 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.031979 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-849pb"] May 28 03:19:41.041156 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.041128 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f74532d-82aa-461a-877f-9be2ca692e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.041287 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.041188 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9f74532d-82aa-461a-877f-9be2ca692e9e-crio-socket\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.041287 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.041227 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f74532d-82aa-461a-877f-9be2ca692e9e-data-volume\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.041287 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.041264 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chhfr\" (UniqueName: \"kubernetes.io/projected/9f74532d-82aa-461a-877f-9be2ca692e9e-kube-api-access-chhfr\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.041403 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.041300 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f74532d-82aa-461a-877f-9be2ca692e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.141624 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.141586 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-chhfr\" (UniqueName: \"kubernetes.io/projected/9f74532d-82aa-461a-877f-9be2ca692e9e-kube-api-access-chhfr\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.141624 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.141627 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f74532d-82aa-461a-877f-9be2ca692e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.141880 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.141654 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f74532d-82aa-461a-877f-9be2ca692e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.141880 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.141692 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9f74532d-82aa-461a-877f-9be2ca692e9e-crio-socket\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.141880 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.141724 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f74532d-82aa-461a-877f-9be2ca692e9e-data-volume\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.141880 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.141824 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9f74532d-82aa-461a-877f-9be2ca692e9e-crio-socket\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.142089 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.142070 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f74532d-82aa-461a-877f-9be2ca692e9e-data-volume\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.142351 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.142317 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f74532d-82aa-461a-877f-9be2ca692e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.144104 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.144085 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f74532d-82aa-461a-877f-9be2ca692e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.157307 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.157271 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chhfr\" (UniqueName: \"kubernetes.io/projected/9f74532d-82aa-461a-877f-9be2ca692e9e-kube-api-access-chhfr\") pod \"insights-runtime-extractor-849pb\" (UID: \"9f74532d-82aa-461a-877f-9be2ca692e9e\") " pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.279413 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.279316 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-849pb" May 28 03:19:41.419920 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:41.419887 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-849pb"] May 28 03:19:41.422978 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:19:41.422936 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f74532d_82aa_461a_877f_9be2ca692e9e.slice/crio-3af48ac6f9d68027ca4e3470d645af50599a2bf06a2dac7af310b904478014ca WatchSource:0}: Error finding container 3af48ac6f9d68027ca4e3470d645af50599a2bf06a2dac7af310b904478014ca: Status 404 returned error can't find the container with id 3af48ac6f9d68027ca4e3470d645af50599a2bf06a2dac7af310b904478014ca May 28 03:19:42.371410 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:42.371330 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kcl9f" May 28 03:19:42.401672 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:42.401634 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-849pb" event={"ID":"9f74532d-82aa-461a-877f-9be2ca692e9e","Type":"ContainerStarted","Data":"76aa9194ad39c5bc245adc68c44e9b6185cbaf99fe14956e79e388b59639b21a"} May 28 03:19:42.401672 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:42.401673 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-849pb" event={"ID":"9f74532d-82aa-461a-877f-9be2ca692e9e","Type":"ContainerStarted","Data":"e4d8ed54c2d1d253e9d919b87749ec27d4a9935b6f9635cdf01d5fe31c48d21a"} May 28 03:19:42.401882 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:42.401683 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-849pb" event={"ID":"9f74532d-82aa-461a-877f-9be2ca692e9e","Type":"ContainerStarted","Data":"3af48ac6f9d68027ca4e3470d645af50599a2bf06a2dac7af310b904478014ca"} May 28 03:19:44.408709 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:44.408674 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-849pb" event={"ID":"9f74532d-82aa-461a-877f-9be2ca692e9e","Type":"ContainerStarted","Data":"c434aed547862206bad1aa6c1068ec75473f463ff67e0f6018e730a4c28a0bad"} May 28 03:19:44.427429 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:44.427380 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-849pb" podStartSLOduration=1.952651267 podStartE2EDuration="4.427363791s" podCreationTimestamp="2026-05-28 03:19:40 +0000 UTC" firstStartedPulling="2026-05-28 03:19:41.493005442 +0000 UTC m=+174.203651382" lastFinishedPulling="2026-05-28 03:19:43.967717966 +0000 UTC m=+176.678363906" observedRunningTime="2026-05-28 03:19:44.427104318 +0000 UTC m=+177.137750279" watchObservedRunningTime="2026-05-28 03:19:44.427363791 +0000 UTC m=+177.138009793" May 28 03:19:49.854904 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:49.854866 2555 patch_prober.go:28] interesting pod/image-registry-6fdb7dd57c-lm4nk container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} May 28 03:19:49.855270 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:49.854928 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" podUID="5a87c119-6651-4c45-87be-302887a66bff" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 28 03:19:51.362790 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:51.362756 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:19:52.038104 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:52.038068 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" podUID="5cc4ec5c-3375-4364-96b0-6b8546bcf156" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 28 03:19:53.517956 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.517918 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rh2gp"] May 28 03:19:53.521451 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.521430 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.530136 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.530089 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 28 03:19:53.530136 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.530089 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 28 03:19:53.530351 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.530154 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 28 03:19:53.530351 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.530165 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 28 03:19:53.531651 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.531630 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 28 03:19:53.531791 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.531736 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 28 03:19:53.531876 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.531858 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7qxsc\"" May 28 03:19:53.639785 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.639746 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-tls\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.639785 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.639792 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1dce139a-e95b-498c-abdd-81dcc70e0d2b-root\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.640011 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.639812 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1dce139a-e95b-498c-abdd-81dcc70e0d2b-sys\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.640011 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.639869 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.640011 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.639941 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-textfile\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.640011 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.639981 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-wtmp\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.640186 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.640013 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dce139a-e95b-498c-abdd-81dcc70e0d2b-metrics-client-ca\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.640186 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.640031 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-accelerators-collector-config\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.640186 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.640048 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nkmv\" (UniqueName: \"kubernetes.io/projected/1dce139a-e95b-498c-abdd-81dcc70e0d2b-kube-api-access-6nkmv\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.740625 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.740581 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-wtmp\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.740827 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.740642 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dce139a-e95b-498c-abdd-81dcc70e0d2b-metrics-client-ca\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.740827 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.740660 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-accelerators-collector-config\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.740827 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.740678 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6nkmv\" (UniqueName: \"kubernetes.io/projected/1dce139a-e95b-498c-abdd-81dcc70e0d2b-kube-api-access-6nkmv\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.740827 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.740710 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-tls\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.740827 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.740742 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1dce139a-e95b-498c-abdd-81dcc70e0d2b-root\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.740827 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.740765 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1dce139a-e95b-498c-abdd-81dcc70e0d2b-sys\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.740827 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.740766 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-wtmp\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.740827 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.740793 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.741177 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.740836 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1dce139a-e95b-498c-abdd-81dcc70e0d2b-sys\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.741177 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.740841 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-textfile\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.741177 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.740835 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1dce139a-e95b-498c-abdd-81dcc70e0d2b-root\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.741314 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.741200 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-textfile\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.741411 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.741387 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1dce139a-e95b-498c-abdd-81dcc70e0d2b-metrics-client-ca\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.741459 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.741394 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-accelerators-collector-config\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.743201 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.743175 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.743365 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.743338 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1dce139a-e95b-498c-abdd-81dcc70e0d2b-node-exporter-tls\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.753431 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.753409 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nkmv\" (UniqueName: \"kubernetes.io/projected/1dce139a-e95b-498c-abdd-81dcc70e0d2b-kube-api-access-6nkmv\") pod \"node-exporter-rh2gp\" (UID: \"1dce139a-e95b-498c-abdd-81dcc70e0d2b\") " pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.831642 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:53.831568 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rh2gp" May 28 03:19:53.840955 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:19:53.840926 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dce139a_e95b_498c_abdd_81dcc70e0d2b.slice/crio-db1814450516b8c7f0bcb2787dbcc9d3842c454e256b7e47d06e35703a8b64c3 WatchSource:0}: Error finding container db1814450516b8c7f0bcb2787dbcc9d3842c454e256b7e47d06e35703a8b64c3: Status 404 returned error can't find the container with id db1814450516b8c7f0bcb2787dbcc9d3842c454e256b7e47d06e35703a8b64c3 May 28 03:19:54.437056 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:54.437012 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rh2gp" event={"ID":"1dce139a-e95b-498c-abdd-81dcc70e0d2b","Type":"ContainerStarted","Data":"db1814450516b8c7f0bcb2787dbcc9d3842c454e256b7e47d06e35703a8b64c3"} May 28 03:19:55.441145 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:55.441105 2555 generic.go:358] "Generic (PLEG): container finished" podID="1dce139a-e95b-498c-abdd-81dcc70e0d2b" containerID="c1f3530bc29254f5655ded338ae64f850ea263a98ce80ca59c967ed7c380283f" exitCode=0 May 28 03:19:55.441557 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:55.441195 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rh2gp" event={"ID":"1dce139a-e95b-498c-abdd-81dcc70e0d2b","Type":"ContainerDied","Data":"c1f3530bc29254f5655ded338ae64f850ea263a98ce80ca59c967ed7c380283f"} May 28 03:19:56.445536 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:56.445479 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rh2gp" event={"ID":"1dce139a-e95b-498c-abdd-81dcc70e0d2b","Type":"ContainerStarted","Data":"19e08123093cd90d83f1565c5466b49750435bcdaa65fd7400c5e2730ea49a28"} May 28 03:19:56.445919 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:56.445543 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rh2gp" event={"ID":"1dce139a-e95b-498c-abdd-81dcc70e0d2b","Type":"ContainerStarted","Data":"eacdcfa6dc04dd3b322549612c607a461acbbf4abeb32ec23fa622a8bb1a90f2"} May 28 03:19:56.475264 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:19:56.475209 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rh2gp" podStartSLOduration=2.503598165 podStartE2EDuration="3.475192875s" podCreationTimestamp="2026-05-28 03:19:53 +0000 UTC" firstStartedPulling="2026-05-28 03:19:53.842836892 +0000 UTC m=+186.553482832" lastFinishedPulling="2026-05-28 03:19:54.814431585 +0000 UTC m=+187.525077542" observedRunningTime="2026-05-28 03:19:56.474345003 +0000 UTC m=+189.184990965" watchObservedRunningTime="2026-05-28 03:19:56.475192875 +0000 UTC m=+189.185838837" May 28 03:20:02.038852 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:02.038807 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" podUID="5cc4ec5c-3375-4364-96b0-6b8546bcf156" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 28 03:20:03.508464 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:03.508428 2555 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6fdb7dd57c-lm4nk"] May 28 03:20:12.038872 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:12.038830 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" podUID="5cc4ec5c-3375-4364-96b0-6b8546bcf156" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 28 03:20:12.039248 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:12.038925 2555 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" May 28 03:20:12.039428 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:12.039397 2555 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"ef5ba60be097d88f43ce3a7242a9fcbff017e0750148d8ec42aa822e3abd65cd"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" containerMessage="Container service-proxy failed liveness probe, will be restarted" May 28 03:20:12.039469 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:12.039449 2555 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" podUID="5cc4ec5c-3375-4364-96b0-6b8546bcf156" containerName="service-proxy" containerID="cri-o://ef5ba60be097d88f43ce3a7242a9fcbff017e0750148d8ec42aa822e3abd65cd" gracePeriod=30 May 28 03:20:12.487332 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:12.487297 2555 generic.go:358] "Generic (PLEG): container finished" podID="5cc4ec5c-3375-4364-96b0-6b8546bcf156" containerID="ef5ba60be097d88f43ce3a7242a9fcbff017e0750148d8ec42aa822e3abd65cd" exitCode=2 May 28 03:20:12.487546 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:12.487377 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" event={"ID":"5cc4ec5c-3375-4364-96b0-6b8546bcf156","Type":"ContainerDied","Data":"ef5ba60be097d88f43ce3a7242a9fcbff017e0750148d8ec42aa822e3abd65cd"} May 28 03:20:12.487546 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:12.487422 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f7c789ff4-2ps4n" event={"ID":"5cc4ec5c-3375-4364-96b0-6b8546bcf156","Type":"ContainerStarted","Data":"85e8f7698bed540628ddb88585049843a00dcc8fdbc2540b684b689d52ad7aef"} May 28 03:20:28.527296 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.527225 2555 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" podUID="5a87c119-6651-4c45-87be-302887a66bff" containerName="registry" containerID="cri-o://11804e82aa701954d2e09f66e52105578c2414605585f846ff307dba094679e2" gracePeriod=30 May 28 03:20:28.769580 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.769554 2555 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:20:28.932773 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.932720 2555 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5a87c119-6651-4c45-87be-302887a66bff-image-registry-private-configuration\") pod \"5a87c119-6651-4c45-87be-302887a66bff\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " May 28 03:20:28.932981 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.932797 2555 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5a87c119-6651-4c45-87be-302887a66bff-installation-pull-secrets\") pod \"5a87c119-6651-4c45-87be-302887a66bff\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " May 28 03:20:28.932981 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.932824 2555 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5w4vd\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-kube-api-access-5w4vd\") pod \"5a87c119-6651-4c45-87be-302887a66bff\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " May 28 03:20:28.932981 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.932848 2555 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-bound-sa-token\") pod \"5a87c119-6651-4c45-87be-302887a66bff\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " May 28 03:20:28.932981 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.932883 2555 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls\") pod \"5a87c119-6651-4c45-87be-302887a66bff\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " May 28 03:20:28.932981 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.932911 2555 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5a87c119-6651-4c45-87be-302887a66bff-registry-certificates\") pod \"5a87c119-6651-4c45-87be-302887a66bff\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " May 28 03:20:28.932981 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.932964 2555 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5a87c119-6651-4c45-87be-302887a66bff-ca-trust-extracted\") pod \"5a87c119-6651-4c45-87be-302887a66bff\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " May 28 03:20:28.933271 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.933002 2555 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a87c119-6651-4c45-87be-302887a66bff-trusted-ca\") pod \"5a87c119-6651-4c45-87be-302887a66bff\" (UID: \"5a87c119-6651-4c45-87be-302887a66bff\") " May 28 03:20:28.933656 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.933446 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a87c119-6651-4c45-87be-302887a66bff-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5a87c119-6651-4c45-87be-302887a66bff" (UID: "5a87c119-6651-4c45-87be-302887a66bff"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 03:20:28.933656 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.933644 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a87c119-6651-4c45-87be-302887a66bff-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5a87c119-6651-4c45-87be-302887a66bff" (UID: "5a87c119-6651-4c45-87be-302887a66bff"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 03:20:28.935556 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.935522 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5a87c119-6651-4c45-87be-302887a66bff" (UID: "5a87c119-6651-4c45-87be-302887a66bff"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 03:20:28.935667 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.935606 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a87c119-6651-4c45-87be-302887a66bff-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5a87c119-6651-4c45-87be-302887a66bff" (UID: "5a87c119-6651-4c45-87be-302887a66bff"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 03:20:28.935667 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.935629 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-kube-api-access-5w4vd" (OuterVolumeSpecName: "kube-api-access-5w4vd") pod "5a87c119-6651-4c45-87be-302887a66bff" (UID: "5a87c119-6651-4c45-87be-302887a66bff"). InnerVolumeSpecName "kube-api-access-5w4vd". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 03:20:28.935746 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.935663 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5a87c119-6651-4c45-87be-302887a66bff" (UID: "5a87c119-6651-4c45-87be-302887a66bff"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 03:20:28.935746 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.935730 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a87c119-6651-4c45-87be-302887a66bff-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5a87c119-6651-4c45-87be-302887a66bff" (UID: "5a87c119-6651-4c45-87be-302887a66bff"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 03:20:28.942200 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:28.942162 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a87c119-6651-4c45-87be-302887a66bff-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5a87c119-6651-4c45-87be-302887a66bff" (UID: "5a87c119-6651-4c45-87be-302887a66bff"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 03:20:29.034406 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.034355 2555 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5a87c119-6651-4c45-87be-302887a66bff-installation-pull-secrets\") on node \"ip-10-0-141-36.ec2.internal\" DevicePath \"\"" May 28 03:20:29.034406 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.034398 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w4vd\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-kube-api-access-5w4vd\") on node \"ip-10-0-141-36.ec2.internal\" DevicePath \"\"" May 28 03:20:29.034406 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.034409 2555 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-bound-sa-token\") on node \"ip-10-0-141-36.ec2.internal\" DevicePath \"\"" May 28 03:20:29.034406 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.034418 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5a87c119-6651-4c45-87be-302887a66bff-registry-tls\") on node \"ip-10-0-141-36.ec2.internal\" DevicePath \"\"" May 28 03:20:29.034406 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.034427 2555 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5a87c119-6651-4c45-87be-302887a66bff-registry-certificates\") on node \"ip-10-0-141-36.ec2.internal\" DevicePath \"\"" May 28 03:20:29.034727 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.034437 2555 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5a87c119-6651-4c45-87be-302887a66bff-ca-trust-extracted\") on node \"ip-10-0-141-36.ec2.internal\" DevicePath \"\"" May 28 03:20:29.034727 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.034446 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5a87c119-6651-4c45-87be-302887a66bff-trusted-ca\") on node \"ip-10-0-141-36.ec2.internal\" DevicePath \"\"" May 28 03:20:29.034727 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.034455 2555 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5a87c119-6651-4c45-87be-302887a66bff-image-registry-private-configuration\") on node \"ip-10-0-141-36.ec2.internal\" DevicePath \"\"" May 28 03:20:29.534337 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.534299 2555 generic.go:358] "Generic (PLEG): container finished" podID="5a87c119-6651-4c45-87be-302887a66bff" containerID="11804e82aa701954d2e09f66e52105578c2414605585f846ff307dba094679e2" exitCode=0 May 28 03:20:29.534767 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.534359 2555 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" May 28 03:20:29.534767 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.534367 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" event={"ID":"5a87c119-6651-4c45-87be-302887a66bff","Type":"ContainerDied","Data":"11804e82aa701954d2e09f66e52105578c2414605585f846ff307dba094679e2"} May 28 03:20:29.534767 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.534402 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fdb7dd57c-lm4nk" event={"ID":"5a87c119-6651-4c45-87be-302887a66bff","Type":"ContainerDied","Data":"bdbc0189b653fd2f3c177331c7f95569c288895da75172cb64e44ae7d8243b2a"} May 28 03:20:29.534767 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.534419 2555 scope.go:117] "RemoveContainer" containerID="11804e82aa701954d2e09f66e52105578c2414605585f846ff307dba094679e2" May 28 03:20:29.543681 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.543664 2555 scope.go:117] "RemoveContainer" containerID="11804e82aa701954d2e09f66e52105578c2414605585f846ff307dba094679e2" May 28 03:20:29.543966 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:20:29.543943 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11804e82aa701954d2e09f66e52105578c2414605585f846ff307dba094679e2\": container with ID starting with 11804e82aa701954d2e09f66e52105578c2414605585f846ff307dba094679e2 not found: ID does not exist" containerID="11804e82aa701954d2e09f66e52105578c2414605585f846ff307dba094679e2" May 28 03:20:29.544014 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.543974 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11804e82aa701954d2e09f66e52105578c2414605585f846ff307dba094679e2"} err="failed to get container status \"11804e82aa701954d2e09f66e52105578c2414605585f846ff307dba094679e2\": rpc error: code = NotFound desc = could not find container \"11804e82aa701954d2e09f66e52105578c2414605585f846ff307dba094679e2\": container with ID starting with 11804e82aa701954d2e09f66e52105578c2414605585f846ff307dba094679e2 not found: ID does not exist" May 28 03:20:29.554408 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.554377 2555 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6fdb7dd57c-lm4nk"] May 28 03:20:29.557977 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.557947 2555 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6fdb7dd57c-lm4nk"] May 28 03:20:29.878123 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:29.878043 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a87c119-6651-4c45-87be-302887a66bff" path="/var/lib/kubelet/pods/5a87c119-6651-4c45-87be-302887a66bff/volumes" May 28 03:20:59.866076 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:59.866031 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:20:59.868459 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:20:59.868436 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8725dbf-86ef-41e7-845a-5b81d2343b2b-metrics-certs\") pod \"network-metrics-daemon-mkdkv\" (UID: \"f8725dbf-86ef-41e7-845a-5b81d2343b2b\") " pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:21:00.078642 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:00.078612 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vzb4w\"" May 28 03:21:00.086277 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:00.086249 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkdkv" May 28 03:21:00.212924 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:00.212890 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mkdkv"] May 28 03:21:00.216596 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:21:00.216563 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8725dbf_86ef_41e7_845a_5b81d2343b2b.slice/crio-b88b3a1fc75315e75473a7372a9035cb25e8da655a05d18cd7b9ddde2eb1269a WatchSource:0}: Error finding container b88b3a1fc75315e75473a7372a9035cb25e8da655a05d18cd7b9ddde2eb1269a: Status 404 returned error can't find the container with id b88b3a1fc75315e75473a7372a9035cb25e8da655a05d18cd7b9ddde2eb1269a May 28 03:21:00.619287 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:00.619250 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mkdkv" event={"ID":"f8725dbf-86ef-41e7-845a-5b81d2343b2b","Type":"ContainerStarted","Data":"b88b3a1fc75315e75473a7372a9035cb25e8da655a05d18cd7b9ddde2eb1269a"} May 28 03:21:01.624605 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:01.624571 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mkdkv" event={"ID":"f8725dbf-86ef-41e7-845a-5b81d2343b2b","Type":"ContainerStarted","Data":"25fff0ce9cce180bd6f30668b8908ebc90618586bfab211f2104a248eeb848b6"} May 28 03:21:01.624605 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:01.624609 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mkdkv" event={"ID":"f8725dbf-86ef-41e7-845a-5b81d2343b2b","Type":"ContainerStarted","Data":"556b9273dad22349a4333fe318752b7b9e58966070c35a93fc6a21d4a73df0e6"} May 28 03:21:01.642697 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:01.642652 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mkdkv" podStartSLOduration=252.464951541 podStartE2EDuration="4m13.642636151s" podCreationTimestamp="2026-05-28 03:16:48 +0000 UTC" firstStartedPulling="2026-05-28 03:21:00.218430699 +0000 UTC m=+252.929076639" lastFinishedPulling="2026-05-28 03:21:01.396115299 +0000 UTC m=+254.106761249" observedRunningTime="2026-05-28 03:21:01.642454267 +0000 UTC m=+254.353100229" watchObservedRunningTime="2026-05-28 03:21:01.642636151 +0000 UTC m=+254.353282112" May 28 03:21:28.339911 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:21:28.339855 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" podUID="7497ac51-1873-4881-9f5d-fd40ca01efd9" May 28 03:21:28.696269 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:28.696240 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:21:31.803103 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:31.803044 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:21:31.805432 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:31.805410 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7497ac51-1873-4881-9f5d-fd40ca01efd9-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-pv6vg\" (UID: \"7497ac51-1873-4881-9f5d-fd40ca01efd9\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:21:31.999947 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:31.999918 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-22499\"" May 28 03:21:32.007429 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:32.007393 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" May 28 03:21:32.142645 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:32.142587 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg"] May 28 03:21:32.146346 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:21:32.146317 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7497ac51_1873_4881_9f5d_fd40ca01efd9.slice/crio-1910163e422595180a7b3f951c96d657cf1b33b7a65bd8b3aba8a27c6d295cc4 WatchSource:0}: Error finding container 1910163e422595180a7b3f951c96d657cf1b33b7a65bd8b3aba8a27c6d295cc4: Status 404 returned error can't find the container with id 1910163e422595180a7b3f951c96d657cf1b33b7a65bd8b3aba8a27c6d295cc4 May 28 03:21:32.707909 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:32.707872 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" event={"ID":"7497ac51-1873-4881-9f5d-fd40ca01efd9","Type":"ContainerStarted","Data":"1910163e422595180a7b3f951c96d657cf1b33b7a65bd8b3aba8a27c6d295cc4"} May 28 03:21:34.714390 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:34.714350 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" event={"ID":"7497ac51-1873-4881-9f5d-fd40ca01efd9","Type":"ContainerStarted","Data":"45596219c5702746be1c218657283cfb0c62497223dd4839b0617fd804a71d4b"} May 28 03:21:34.730646 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:34.730589 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-pv6vg" podStartSLOduration=278.328653225 podStartE2EDuration="4m40.730571958s" podCreationTimestamp="2026-05-28 03:16:54 +0000 UTC" firstStartedPulling="2026-05-28 03:21:32.14806832 +0000 UTC m=+284.858714260" lastFinishedPulling="2026-05-28 03:21:34.549987035 +0000 UTC m=+287.260632993" observedRunningTime="2026-05-28 03:21:34.729365024 +0000 UTC m=+287.440010986" watchObservedRunningTime="2026-05-28 03:21:34.730571958 +0000 UTC m=+287.441217920" May 28 03:21:47.763637 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:47.763611 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:21:47.764128 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:47.763641 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:21:47.766125 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:21:47.766109 2555 kubelet.go:1628] "Image garbage collection succeeded" May 28 03:24:23.506964 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.506881 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-cn87g"] May 28 03:24:23.507439 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.507101 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a87c119-6651-4c45-87be-302887a66bff" containerName="registry" May 28 03:24:23.507439 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.507112 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a87c119-6651-4c45-87be-302887a66bff" containerName="registry" May 28 03:24:23.507439 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.507163 2555 memory_manager.go:356] "RemoveStaleState removing state" podUID="5a87c119-6651-4c45-87be-302887a66bff" containerName="registry" May 28 03:24:23.509833 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.509815 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-cn87g" May 28 03:24:23.512279 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.512250 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" May 28 03:24:23.512441 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.512250 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-9574r\"" May 28 03:24:23.512441 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.512251 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" May 28 03:24:23.517171 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.517141 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-cn87g"] May 28 03:24:23.543219 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.543177 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de96ff7f-b981-41bc-8efc-13783ec90395-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-cn87g\" (UID: \"de96ff7f-b981-41bc-8efc-13783ec90395\") " pod="cert-manager/cert-manager-webhook-587ccfb98-cn87g" May 28 03:24:23.543400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.543238 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnzm7\" (UniqueName: \"kubernetes.io/projected/de96ff7f-b981-41bc-8efc-13783ec90395-kube-api-access-mnzm7\") pod \"cert-manager-webhook-587ccfb98-cn87g\" (UID: \"de96ff7f-b981-41bc-8efc-13783ec90395\") " pod="cert-manager/cert-manager-webhook-587ccfb98-cn87g" May 28 03:24:23.644561 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.644517 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de96ff7f-b981-41bc-8efc-13783ec90395-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-cn87g\" (UID: \"de96ff7f-b981-41bc-8efc-13783ec90395\") " pod="cert-manager/cert-manager-webhook-587ccfb98-cn87g" May 28 03:24:23.644663 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.644590 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mnzm7\" (UniqueName: \"kubernetes.io/projected/de96ff7f-b981-41bc-8efc-13783ec90395-kube-api-access-mnzm7\") pod \"cert-manager-webhook-587ccfb98-cn87g\" (UID: \"de96ff7f-b981-41bc-8efc-13783ec90395\") " pod="cert-manager/cert-manager-webhook-587ccfb98-cn87g" May 28 03:24:23.653482 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.653447 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de96ff7f-b981-41bc-8efc-13783ec90395-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-cn87g\" (UID: \"de96ff7f-b981-41bc-8efc-13783ec90395\") " pod="cert-manager/cert-manager-webhook-587ccfb98-cn87g" May 28 03:24:23.654073 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.654051 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnzm7\" (UniqueName: \"kubernetes.io/projected/de96ff7f-b981-41bc-8efc-13783ec90395-kube-api-access-mnzm7\") pod \"cert-manager-webhook-587ccfb98-cn87g\" (UID: \"de96ff7f-b981-41bc-8efc-13783ec90395\") " pod="cert-manager/cert-manager-webhook-587ccfb98-cn87g" May 28 03:24:23.819782 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.819677 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-cn87g" May 28 03:24:23.948694 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.948662 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-cn87g"] May 28 03:24:23.952163 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:24:23.952134 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde96ff7f_b981_41bc_8efc_13783ec90395.slice/crio-d507e20a5f5c00a112295e2aa65f87634e823bf658736d41e72d21ce9bbff65d WatchSource:0}: Error finding container d507e20a5f5c00a112295e2aa65f87634e823bf658736d41e72d21ce9bbff65d: Status 404 returned error can't find the container with id d507e20a5f5c00a112295e2aa65f87634e823bf658736d41e72d21ce9bbff65d May 28 03:24:23.954014 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:23.953993 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 03:24:24.160812 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:24.160775 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-cn87g" event={"ID":"de96ff7f-b981-41bc-8efc-13783ec90395","Type":"ContainerStarted","Data":"d507e20a5f5c00a112295e2aa65f87634e823bf658736d41e72d21ce9bbff65d"} May 28 03:24:27.172850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:27.172812 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-cn87g" event={"ID":"de96ff7f-b981-41bc-8efc-13783ec90395","Type":"ContainerStarted","Data":"0db8b5b8226853a20938b516893bbfec03f6faced53076463ae8e99b71d28974"} May 28 03:24:27.173241 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:27.172889 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-587ccfb98-cn87g" May 28 03:24:27.191835 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:27.191782 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-587ccfb98-cn87g" podStartSLOduration=1.961181555 podStartE2EDuration="4.191766193s" podCreationTimestamp="2026-05-28 03:24:23 +0000 UTC" firstStartedPulling="2026-05-28 03:24:23.954123751 +0000 UTC m=+456.664769691" lastFinishedPulling="2026-05-28 03:24:26.184708371 +0000 UTC m=+458.895354329" observedRunningTime="2026-05-28 03:24:27.18987316 +0000 UTC m=+459.900519122" watchObservedRunningTime="2026-05-28 03:24:27.191766193 +0000 UTC m=+459.902412154" May 28 03:24:33.177968 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:33.177936 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-587ccfb98-cn87g" May 28 03:24:52.195206 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.195167 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh"] May 28 03:24:52.199475 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.199454 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" May 28 03:24:52.203753 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.203733 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 28 03:24:52.204509 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.204470 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 28 03:24:52.204628 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.204582 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-l2nrl\"" May 28 03:24:52.204710 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.204485 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 28 03:24:52.204951 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.204920 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 28 03:24:52.216470 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.216440 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh"] May 28 03:24:52.355483 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.355448 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac7ef498-9c08-40af-a053-34db052d2f91-webhook-cert\") pod \"opendatahub-operator-controller-manager-77cd789dcd-2fnxh\" (UID: \"ac7ef498-9c08-40af-a053-34db052d2f91\") " pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" May 28 03:24:52.355723 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.355529 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ac7ef498-9c08-40af-a053-34db052d2f91-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77cd789dcd-2fnxh\" (UID: \"ac7ef498-9c08-40af-a053-34db052d2f91\") " pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" May 28 03:24:52.355723 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.355549 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7zbj\" (UniqueName: \"kubernetes.io/projected/ac7ef498-9c08-40af-a053-34db052d2f91-kube-api-access-v7zbj\") pod \"opendatahub-operator-controller-manager-77cd789dcd-2fnxh\" (UID: \"ac7ef498-9c08-40af-a053-34db052d2f91\") " pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" May 28 03:24:52.456380 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.456294 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ac7ef498-9c08-40af-a053-34db052d2f91-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77cd789dcd-2fnxh\" (UID: \"ac7ef498-9c08-40af-a053-34db052d2f91\") " pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" May 28 03:24:52.456380 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.456330 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v7zbj\" (UniqueName: \"kubernetes.io/projected/ac7ef498-9c08-40af-a053-34db052d2f91-kube-api-access-v7zbj\") pod \"opendatahub-operator-controller-manager-77cd789dcd-2fnxh\" (UID: \"ac7ef498-9c08-40af-a053-34db052d2f91\") " pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" May 28 03:24:52.456380 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.456379 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac7ef498-9c08-40af-a053-34db052d2f91-webhook-cert\") pod \"opendatahub-operator-controller-manager-77cd789dcd-2fnxh\" (UID: \"ac7ef498-9c08-40af-a053-34db052d2f91\") " pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" May 28 03:24:52.458931 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.458901 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ac7ef498-9c08-40af-a053-34db052d2f91-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77cd789dcd-2fnxh\" (UID: \"ac7ef498-9c08-40af-a053-34db052d2f91\") " pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" May 28 03:24:52.459053 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.458953 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac7ef498-9c08-40af-a053-34db052d2f91-webhook-cert\") pod \"opendatahub-operator-controller-manager-77cd789dcd-2fnxh\" (UID: \"ac7ef498-9c08-40af-a053-34db052d2f91\") " pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" May 28 03:24:52.466860 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.466834 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7zbj\" (UniqueName: \"kubernetes.io/projected/ac7ef498-9c08-40af-a053-34db052d2f91-kube-api-access-v7zbj\") pod \"opendatahub-operator-controller-manager-77cd789dcd-2fnxh\" (UID: \"ac7ef498-9c08-40af-a053-34db052d2f91\") " pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" May 28 03:24:52.509513 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.509452 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" May 28 03:24:52.647663 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:52.647626 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh"] May 28 03:24:52.652927 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:24:52.652898 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac7ef498_9c08_40af_a053_34db052d2f91.slice/crio-5f07253777696da5dc9bf69c41e6c6edc1821d7370afa5ded31cd2fdb666a4e1 WatchSource:0}: Error finding container 5f07253777696da5dc9bf69c41e6c6edc1821d7370afa5ded31cd2fdb666a4e1: Status 404 returned error can't find the container with id 5f07253777696da5dc9bf69c41e6c6edc1821d7370afa5ded31cd2fdb666a4e1 May 28 03:24:53.243584 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:53.243542 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" event={"ID":"ac7ef498-9c08-40af-a053-34db052d2f91","Type":"ContainerStarted","Data":"5f07253777696da5dc9bf69c41e6c6edc1821d7370afa5ded31cd2fdb666a4e1"} May 28 03:24:55.250588 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:55.250546 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" event={"ID":"ac7ef498-9c08-40af-a053-34db052d2f91","Type":"ContainerStarted","Data":"cbe796f6496379dcbe2347b506e33b9dfa1cd3ef8a3e480502951c6f8fe3b64c"} May 28 03:24:55.251054 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:55.250655 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" May 28 03:24:55.302137 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:24:55.302087 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" podStartSLOduration=0.86998549 podStartE2EDuration="3.302069979s" podCreationTimestamp="2026-05-28 03:24:52 +0000 UTC" firstStartedPulling="2026-05-28 03:24:52.656222155 +0000 UTC m=+485.366868098" lastFinishedPulling="2026-05-28 03:24:55.088306644 +0000 UTC m=+487.798952587" observedRunningTime="2026-05-28 03:24:55.3013136 +0000 UTC m=+488.011959563" watchObservedRunningTime="2026-05-28 03:24:55.302069979 +0000 UTC m=+488.012715940" May 28 03:25:06.255633 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:06.255599 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-77cd789dcd-2fnxh" May 28 03:25:10.157713 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.157677 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h"] May 28 03:25:10.160689 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.160667 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" May 28 03:25:10.164105 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.164076 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" May 28 03:25:10.164262 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.164136 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2d89b\"" May 28 03:25:10.164262 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.164076 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 28 03:25:10.164262 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.164208 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" May 28 03:25:10.164262 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.164103 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 28 03:25:10.178328 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.178280 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h"] May 28 03:25:10.294711 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.294674 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92-tls-certs\") pod \"kube-auth-proxy-5b78c488db-7qn7h\" (UID: \"34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92\") " pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" May 28 03:25:10.294905 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.294728 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-974f6\" (UniqueName: \"kubernetes.io/projected/34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92-kube-api-access-974f6\") pod \"kube-auth-proxy-5b78c488db-7qn7h\" (UID: \"34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92\") " pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" May 28 03:25:10.294905 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.294807 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92-tmp\") pod \"kube-auth-proxy-5b78c488db-7qn7h\" (UID: \"34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92\") " pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" May 28 03:25:10.396069 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.396021 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92-tmp\") pod \"kube-auth-proxy-5b78c488db-7qn7h\" (UID: \"34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92\") " pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" May 28 03:25:10.396263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.396089 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92-tls-certs\") pod \"kube-auth-proxy-5b78c488db-7qn7h\" (UID: \"34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92\") " pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" May 28 03:25:10.396263 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.396131 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-974f6\" (UniqueName: \"kubernetes.io/projected/34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92-kube-api-access-974f6\") pod \"kube-auth-proxy-5b78c488db-7qn7h\" (UID: \"34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92\") " pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" May 28 03:25:10.398540 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.398485 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92-tmp\") pod \"kube-auth-proxy-5b78c488db-7qn7h\" (UID: \"34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92\") " pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" May 28 03:25:10.398653 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.398639 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92-tls-certs\") pod \"kube-auth-proxy-5b78c488db-7qn7h\" (UID: \"34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92\") " pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" May 28 03:25:10.411890 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.411821 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-974f6\" (UniqueName: \"kubernetes.io/projected/34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92-kube-api-access-974f6\") pod \"kube-auth-proxy-5b78c488db-7qn7h\" (UID: \"34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92\") " pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" May 28 03:25:10.470451 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.470401 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" May 28 03:25:10.602511 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:10.602458 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h"] May 28 03:25:10.605759 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:25:10.605718 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c4d4c6_6a98_42c0_b0f1_8e4a0ac77f92.slice/crio-c0d3510cfdf17aa56526beb83abfa31c0ea589c8472c9bf296d38b7e561cb775 WatchSource:0}: Error finding container c0d3510cfdf17aa56526beb83abfa31c0ea589c8472c9bf296d38b7e561cb775: Status 404 returned error can't find the container with id c0d3510cfdf17aa56526beb83abfa31c0ea589c8472c9bf296d38b7e561cb775 May 28 03:25:11.294808 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:11.294763 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" event={"ID":"34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92","Type":"ContainerStarted","Data":"c0d3510cfdf17aa56526beb83abfa31c0ea589c8472c9bf296d38b7e561cb775"} May 28 03:25:13.389635 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:13.389596 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-hvzw9"] May 28 03:25:13.392852 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:13.392827 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" May 28 03:25:13.395287 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:13.395258 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" May 28 03:25:13.395442 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:13.395372 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-k7pcb\"" May 28 03:25:13.402322 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:13.402262 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-hvzw9"] May 28 03:25:13.521543 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:13.521479 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km4n2\" (UniqueName: \"kubernetes.io/projected/0ed3886f-9b44-40e8-a0ea-32171686ff9f-kube-api-access-km4n2\") pod \"odh-model-controller-858dbf95b8-hvzw9\" (UID: \"0ed3886f-9b44-40e8-a0ea-32171686ff9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" May 28 03:25:13.521722 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:13.521617 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ed3886f-9b44-40e8-a0ea-32171686ff9f-cert\") pod \"odh-model-controller-858dbf95b8-hvzw9\" (UID: \"0ed3886f-9b44-40e8-a0ea-32171686ff9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" May 28 03:25:13.622914 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:13.622866 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ed3886f-9b44-40e8-a0ea-32171686ff9f-cert\") pod \"odh-model-controller-858dbf95b8-hvzw9\" (UID: \"0ed3886f-9b44-40e8-a0ea-32171686ff9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" May 28 03:25:13.623117 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:13.622931 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-km4n2\" (UniqueName: \"kubernetes.io/projected/0ed3886f-9b44-40e8-a0ea-32171686ff9f-kube-api-access-km4n2\") pod \"odh-model-controller-858dbf95b8-hvzw9\" (UID: \"0ed3886f-9b44-40e8-a0ea-32171686ff9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" May 28 03:25:13.623117 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:25:13.623040 2555 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found May 28 03:25:13.623230 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:25:13.623138 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ed3886f-9b44-40e8-a0ea-32171686ff9f-cert podName:0ed3886f-9b44-40e8-a0ea-32171686ff9f nodeName:}" failed. No retries permitted until 2026-05-28 03:25:14.123114826 +0000 UTC m=+506.833760768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0ed3886f-9b44-40e8-a0ea-32171686ff9f-cert") pod "odh-model-controller-858dbf95b8-hvzw9" (UID: "0ed3886f-9b44-40e8-a0ea-32171686ff9f") : secret "odh-model-controller-webhook-cert" not found May 28 03:25:13.634274 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:13.634239 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-km4n2\" (UniqueName: \"kubernetes.io/projected/0ed3886f-9b44-40e8-a0ea-32171686ff9f-kube-api-access-km4n2\") pod \"odh-model-controller-858dbf95b8-hvzw9\" (UID: \"0ed3886f-9b44-40e8-a0ea-32171686ff9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" May 28 03:25:14.128656 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:14.128604 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ed3886f-9b44-40e8-a0ea-32171686ff9f-cert\") pod \"odh-model-controller-858dbf95b8-hvzw9\" (UID: \"0ed3886f-9b44-40e8-a0ea-32171686ff9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" May 28 03:25:14.131705 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:14.131667 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ed3886f-9b44-40e8-a0ea-32171686ff9f-cert\") pod \"odh-model-controller-858dbf95b8-hvzw9\" (UID: \"0ed3886f-9b44-40e8-a0ea-32171686ff9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" May 28 03:25:14.306868 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:14.306778 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" May 28 03:25:14.308222 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:14.308193 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" event={"ID":"34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92","Type":"ContainerStarted","Data":"a5db59d3b749e0cca7f36fdab17c3d6639c899b240543d2596441ab7b2291fc4"} May 28 03:25:14.328071 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:14.328002 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b78c488db-7qn7h" podStartSLOduration=0.949608217 podStartE2EDuration="4.32798056s" podCreationTimestamp="2026-05-28 03:25:10 +0000 UTC" firstStartedPulling="2026-05-28 03:25:10.607351769 +0000 UTC m=+503.317997710" lastFinishedPulling="2026-05-28 03:25:13.985724114 +0000 UTC m=+506.696370053" observedRunningTime="2026-05-28 03:25:14.326874346 +0000 UTC m=+507.037520309" watchObservedRunningTime="2026-05-28 03:25:14.32798056 +0000 UTC m=+507.038626524" May 28 03:25:14.445668 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:14.445634 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-hvzw9"] May 28 03:25:14.450217 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:25:14.450185 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed3886f_9b44_40e8_a0ea_32171686ff9f.slice/crio-82db6edf5240520cccccdc6b16f01ab2b4d712b29cb0bc1a1c51cb71164f5db9 WatchSource:0}: Error finding container 82db6edf5240520cccccdc6b16f01ab2b4d712b29cb0bc1a1c51cb71164f5db9: Status 404 returned error can't find the container with id 82db6edf5240520cccccdc6b16f01ab2b4d712b29cb0bc1a1c51cb71164f5db9 May 28 03:25:15.312982 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:15.312935 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" event={"ID":"0ed3886f-9b44-40e8-a0ea-32171686ff9f","Type":"ContainerStarted","Data":"82db6edf5240520cccccdc6b16f01ab2b4d712b29cb0bc1a1c51cb71164f5db9"} May 28 03:25:18.323347 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:18.323312 2555 generic.go:358] "Generic (PLEG): container finished" podID="0ed3886f-9b44-40e8-a0ea-32171686ff9f" containerID="17408a55c74ecabcb32dcd74d99c5e20e5454ce8312c6110b57df9978abce142" exitCode=1 May 28 03:25:18.323762 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:18.323412 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" event={"ID":"0ed3886f-9b44-40e8-a0ea-32171686ff9f","Type":"ContainerDied","Data":"17408a55c74ecabcb32dcd74d99c5e20e5454ce8312c6110b57df9978abce142"} May 28 03:25:18.323762 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:18.323684 2555 scope.go:117] "RemoveContainer" containerID="17408a55c74ecabcb32dcd74d99c5e20e5454ce8312c6110b57df9978abce142" May 28 03:25:18.860512 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:18.860389 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-4rpvb"] May 28 03:25:18.863637 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:18.863609 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" May 28 03:25:18.865900 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:18.865870 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" May 28 03:25:18.866038 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:18.865877 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-bfh84\"" May 28 03:25:18.871190 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:18.871125 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/568e1f50-02ca-4f32-b096-78b44cbc6b4d-cert\") pod \"kserve-controller-manager-856948b99f-4rpvb\" (UID: \"568e1f50-02ca-4f32-b096-78b44cbc6b4d\") " pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" May 28 03:25:18.871330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:18.871198 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4th4z\" (UniqueName: \"kubernetes.io/projected/568e1f50-02ca-4f32-b096-78b44cbc6b4d-kube-api-access-4th4z\") pod \"kserve-controller-manager-856948b99f-4rpvb\" (UID: \"568e1f50-02ca-4f32-b096-78b44cbc6b4d\") " pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" May 28 03:25:18.873789 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:18.873757 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-4rpvb"] May 28 03:25:18.972786 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:18.972746 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4th4z\" (UniqueName: \"kubernetes.io/projected/568e1f50-02ca-4f32-b096-78b44cbc6b4d-kube-api-access-4th4z\") pod \"kserve-controller-manager-856948b99f-4rpvb\" (UID: \"568e1f50-02ca-4f32-b096-78b44cbc6b4d\") " pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" May 28 03:25:18.972978 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:18.972844 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/568e1f50-02ca-4f32-b096-78b44cbc6b4d-cert\") pod \"kserve-controller-manager-856948b99f-4rpvb\" (UID: \"568e1f50-02ca-4f32-b096-78b44cbc6b4d\") " pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" May 28 03:25:18.972978 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:25:18.972972 2555 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found May 28 03:25:18.973087 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:25:18.973035 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/568e1f50-02ca-4f32-b096-78b44cbc6b4d-cert podName:568e1f50-02ca-4f32-b096-78b44cbc6b4d nodeName:}" failed. No retries permitted until 2026-05-28 03:25:19.47301428 +0000 UTC m=+512.183660237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/568e1f50-02ca-4f32-b096-78b44cbc6b4d-cert") pod "kserve-controller-manager-856948b99f-4rpvb" (UID: "568e1f50-02ca-4f32-b096-78b44cbc6b4d") : secret "kserve-webhook-server-cert" not found May 28 03:25:18.982243 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:18.982208 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4th4z\" (UniqueName: \"kubernetes.io/projected/568e1f50-02ca-4f32-b096-78b44cbc6b4d-kube-api-access-4th4z\") pod \"kserve-controller-manager-856948b99f-4rpvb\" (UID: \"568e1f50-02ca-4f32-b096-78b44cbc6b4d\") " pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" May 28 03:25:19.327944 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:19.327909 2555 generic.go:358] "Generic (PLEG): container finished" podID="0ed3886f-9b44-40e8-a0ea-32171686ff9f" containerID="2bc0d38fd60a0b4e314f338e097f323122abdd2eca8594ea9a27dfcf91d53d47" exitCode=1 May 28 03:25:19.328420 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:19.327999 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" event={"ID":"0ed3886f-9b44-40e8-a0ea-32171686ff9f","Type":"ContainerDied","Data":"2bc0d38fd60a0b4e314f338e097f323122abdd2eca8594ea9a27dfcf91d53d47"} May 28 03:25:19.328420 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:19.328067 2555 scope.go:117] "RemoveContainer" containerID="17408a55c74ecabcb32dcd74d99c5e20e5454ce8312c6110b57df9978abce142" May 28 03:25:19.328420 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:19.328269 2555 scope.go:117] "RemoveContainer" containerID="2bc0d38fd60a0b4e314f338e097f323122abdd2eca8594ea9a27dfcf91d53d47" May 28 03:25:19.328619 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:25:19.328527 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-hvzw9_opendatahub(0ed3886f-9b44-40e8-a0ea-32171686ff9f)\"" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" podUID="0ed3886f-9b44-40e8-a0ea-32171686ff9f" May 28 03:25:19.476045 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:19.476010 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/568e1f50-02ca-4f32-b096-78b44cbc6b4d-cert\") pod \"kserve-controller-manager-856948b99f-4rpvb\" (UID: \"568e1f50-02ca-4f32-b096-78b44cbc6b4d\") " pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" May 28 03:25:19.478368 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:19.478343 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/568e1f50-02ca-4f32-b096-78b44cbc6b4d-cert\") pod \"kserve-controller-manager-856948b99f-4rpvb\" (UID: \"568e1f50-02ca-4f32-b096-78b44cbc6b4d\") " pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" May 28 03:25:19.775737 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:19.775703 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" May 28 03:25:19.922172 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:19.922148 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-4rpvb"] May 28 03:25:19.923723 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:25:19.923695 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568e1f50_02ca_4f32_b096_78b44cbc6b4d.slice/crio-b31c9512bef5ec84922f239aa53e21b33aae65877fc398b9317beb8d34d94ea8 WatchSource:0}: Error finding container b31c9512bef5ec84922f239aa53e21b33aae65877fc398b9317beb8d34d94ea8: Status 404 returned error can't find the container with id b31c9512bef5ec84922f239aa53e21b33aae65877fc398b9317beb8d34d94ea8 May 28 03:25:20.331750 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:20.331716 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" event={"ID":"568e1f50-02ca-4f32-b096-78b44cbc6b4d","Type":"ContainerStarted","Data":"b31c9512bef5ec84922f239aa53e21b33aae65877fc398b9317beb8d34d94ea8"} May 28 03:25:20.333296 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:20.333277 2555 scope.go:117] "RemoveContainer" containerID="2bc0d38fd60a0b4e314f338e097f323122abdd2eca8594ea9a27dfcf91d53d47" May 28 03:25:20.333465 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:25:20.333449 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-hvzw9_opendatahub(0ed3886f-9b44-40e8-a0ea-32171686ff9f)\"" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" podUID="0ed3886f-9b44-40e8-a0ea-32171686ff9f" May 28 03:25:24.306895 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:24.306859 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" May 28 03:25:24.307289 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:24.307243 2555 scope.go:117] "RemoveContainer" containerID="2bc0d38fd60a0b4e314f338e097f323122abdd2eca8594ea9a27dfcf91d53d47" May 28 03:25:24.307424 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:25:24.307404 2555 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-hvzw9_opendatahub(0ed3886f-9b44-40e8-a0ea-32171686ff9f)\"" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" podUID="0ed3886f-9b44-40e8-a0ea-32171686ff9f" May 28 03:25:24.345758 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:24.345721 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" event={"ID":"568e1f50-02ca-4f32-b096-78b44cbc6b4d","Type":"ContainerStarted","Data":"3f5e780fc5f1ebf16b2fd1548feeb0c0bae529e353a4f2d309837cc8739f86c5"} May 28 03:25:24.345922 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:24.345799 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" May 28 03:25:24.372016 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:24.371963 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" podStartSLOduration=2.122070318 podStartE2EDuration="6.371948291s" podCreationTimestamp="2026-05-28 03:25:18 +0000 UTC" firstStartedPulling="2026-05-28 03:25:19.925184409 +0000 UTC m=+512.635830351" lastFinishedPulling="2026-05-28 03:25:24.175062381 +0000 UTC m=+516.885708324" observedRunningTime="2026-05-28 03:25:24.371946566 +0000 UTC m=+517.082592529" watchObservedRunningTime="2026-05-28 03:25:24.371948291 +0000 UTC m=+517.082594253" May 28 03:25:24.960366 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:24.960102 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2"] May 28 03:25:24.963436 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:24.963410 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2" May 28 03:25:24.965966 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:24.965942 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" May 28 03:25:24.966115 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:24.966015 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" May 28 03:25:24.966316 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:24.966301 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-bl5m5\"" May 28 03:25:24.976932 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:24.976900 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2"] May 28 03:25:25.011581 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:25.011547 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/8ba61363-2512-4f35-8f74-0d5b36c0bc23-operator-config\") pod \"servicemesh-operator3-55f49c5f94-4kdp2\" (UID: \"8ba61363-2512-4f35-8f74-0d5b36c0bc23\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2" May 28 03:25:25.011749 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:25.011622 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz9m6\" (UniqueName: \"kubernetes.io/projected/8ba61363-2512-4f35-8f74-0d5b36c0bc23-kube-api-access-zz9m6\") pod \"servicemesh-operator3-55f49c5f94-4kdp2\" (UID: \"8ba61363-2512-4f35-8f74-0d5b36c0bc23\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2" May 28 03:25:25.112816 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:25.112777 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zz9m6\" (UniqueName: \"kubernetes.io/projected/8ba61363-2512-4f35-8f74-0d5b36c0bc23-kube-api-access-zz9m6\") pod \"servicemesh-operator3-55f49c5f94-4kdp2\" (UID: \"8ba61363-2512-4f35-8f74-0d5b36c0bc23\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2" May 28 03:25:25.113003 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:25.112826 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/8ba61363-2512-4f35-8f74-0d5b36c0bc23-operator-config\") pod \"servicemesh-operator3-55f49c5f94-4kdp2\" (UID: \"8ba61363-2512-4f35-8f74-0d5b36c0bc23\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2" May 28 03:25:25.115273 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:25.115245 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/8ba61363-2512-4f35-8f74-0d5b36c0bc23-operator-config\") pod \"servicemesh-operator3-55f49c5f94-4kdp2\" (UID: \"8ba61363-2512-4f35-8f74-0d5b36c0bc23\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2" May 28 03:25:25.121139 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:25.121116 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz9m6\" (UniqueName: \"kubernetes.io/projected/8ba61363-2512-4f35-8f74-0d5b36c0bc23-kube-api-access-zz9m6\") pod \"servicemesh-operator3-55f49c5f94-4kdp2\" (UID: \"8ba61363-2512-4f35-8f74-0d5b36c0bc23\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2" May 28 03:25:25.273346 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:25.273248 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2" May 28 03:25:25.421681 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:25.421649 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2"] May 28 03:25:25.426104 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:25:25.426076 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba61363_2512_4f35_8f74_0d5b36c0bc23.slice/crio-fa2ea369c82190497a7843c49f65c929364adb695e48157fdfd8c6727949361e WatchSource:0}: Error finding container fa2ea369c82190497a7843c49f65c929364adb695e48157fdfd8c6727949361e: Status 404 returned error can't find the container with id fa2ea369c82190497a7843c49f65c929364adb695e48157fdfd8c6727949361e May 28 03:25:26.354654 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:26.354601 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2" event={"ID":"8ba61363-2512-4f35-8f74-0d5b36c0bc23","Type":"ContainerStarted","Data":"fa2ea369c82190497a7843c49f65c929364adb695e48157fdfd8c6727949361e"} May 28 03:25:30.373506 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:30.373457 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2" event={"ID":"8ba61363-2512-4f35-8f74-0d5b36c0bc23","Type":"ContainerStarted","Data":"9e2eab576fac1c70401a093806e77107a0b7d037c85332cf9d5600c445b7add0"} May 28 03:25:30.373858 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:30.373548 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2" May 28 03:25:30.395483 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:30.395424 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2" podStartSLOduration=2.266705162 podStartE2EDuration="6.395405572s" podCreationTimestamp="2026-05-28 03:25:24 +0000 UTC" firstStartedPulling="2026-05-28 03:25:25.429326635 +0000 UTC m=+518.139972576" lastFinishedPulling="2026-05-28 03:25:29.558027045 +0000 UTC m=+522.268672986" observedRunningTime="2026-05-28 03:25:30.394023356 +0000 UTC m=+523.104669330" watchObservedRunningTime="2026-05-28 03:25:30.395405572 +0000 UTC m=+523.106051536" May 28 03:25:34.307706 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:34.307661 2555 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" May 28 03:25:34.308100 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:34.308085 2555 scope.go:117] "RemoveContainer" containerID="2bc0d38fd60a0b4e314f338e097f323122abdd2eca8594ea9a27dfcf91d53d47" May 28 03:25:35.390530 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:35.390473 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" event={"ID":"0ed3886f-9b44-40e8-a0ea-32171686ff9f","Type":"ContainerStarted","Data":"f5cd2d95f97c1babebbe675183dd2cfbacdabc9b08a4bda726cf0664d9d9f57e"} May 28 03:25:35.390957 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:35.390713 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" May 28 03:25:35.409849 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:35.409790 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" podStartSLOduration=2.196841015 podStartE2EDuration="22.409774404s" podCreationTimestamp="2026-05-28 03:25:13 +0000 UTC" firstStartedPulling="2026-05-28 03:25:14.45190223 +0000 UTC m=+507.162548170" lastFinishedPulling="2026-05-28 03:25:34.664835618 +0000 UTC m=+527.375481559" observedRunningTime="2026-05-28 03:25:35.409242833 +0000 UTC m=+528.119888786" watchObservedRunningTime="2026-05-28 03:25:35.409774404 +0000 UTC m=+528.120420366" May 28 03:25:41.379271 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:41.379190 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-4kdp2" May 28 03:25:46.395614 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:46.395578 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-hvzw9" May 28 03:25:55.354400 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:25:55.354362 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-4rpvb" May 28 03:26:09.822451 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:09.822411 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmjwk"] May 28 03:26:09.828977 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:09.828954 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmjwk" May 28 03:26:09.833469 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:09.833432 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 28 03:26:09.833705 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:09.833687 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-f5vr9\"" May 28 03:26:09.834027 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:09.834006 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 28 03:26:09.845454 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:09.845431 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmjwk"] May 28 03:26:09.923024 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:09.922989 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jjcl\" (UniqueName: \"kubernetes.io/projected/1e93a8c6-fa42-4bb7-a759-35de1be63004-kube-api-access-7jjcl\") pod \"kuadrant-operator-catalog-mmjwk\" (UID: \"1e93a8c6-fa42-4bb7-a759-35de1be63004\") " pod="kuadrant-system/kuadrant-operator-catalog-mmjwk" May 28 03:26:10.024324 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.024279 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7jjcl\" (UniqueName: \"kubernetes.io/projected/1e93a8c6-fa42-4bb7-a759-35de1be63004-kube-api-access-7jjcl\") pod \"kuadrant-operator-catalog-mmjwk\" (UID: \"1e93a8c6-fa42-4bb7-a759-35de1be63004\") " pod="kuadrant-system/kuadrant-operator-catalog-mmjwk" May 28 03:26:10.033277 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.033241 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jjcl\" (UniqueName: \"kubernetes.io/projected/1e93a8c6-fa42-4bb7-a759-35de1be63004-kube-api-access-7jjcl\") pod \"kuadrant-operator-catalog-mmjwk\" (UID: \"1e93a8c6-fa42-4bb7-a759-35de1be63004\") " pod="kuadrant-system/kuadrant-operator-catalog-mmjwk" May 28 03:26:10.138353 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.138268 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmjwk" May 28 03:26:10.158840 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.158813 2555 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmjwk"] May 28 03:26:10.277545 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.277520 2555 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmjwk"] May 28 03:26:10.279686 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:26:10.279654 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e93a8c6_fa42_4bb7_a759_35de1be63004.slice/crio-6d7b95037ed66cda64ba090f8934b09f466a05780f4dd133c28ae0bdcd848432 WatchSource:0}: Error finding container 6d7b95037ed66cda64ba090f8934b09f466a05780f4dd133c28ae0bdcd848432: Status 404 returned error can't find the container with id 6d7b95037ed66cda64ba090f8934b09f466a05780f4dd133c28ae0bdcd848432 May 28 03:26:10.369668 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.369626 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rlfk4"] May 28 03:26:10.374034 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.374006 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rlfk4" May 28 03:26:10.380307 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.380267 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rlfk4"] May 28 03:26:10.426914 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.426878 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nvgb\" (UniqueName: \"kubernetes.io/projected/993a9ccc-e9ed-402b-b76f-92a749da0996-kube-api-access-6nvgb\") pod \"kuadrant-operator-catalog-rlfk4\" (UID: \"993a9ccc-e9ed-402b-b76f-92a749da0996\") " pod="kuadrant-system/kuadrant-operator-catalog-rlfk4" May 28 03:26:10.506823 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.506789 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mmjwk" event={"ID":"1e93a8c6-fa42-4bb7-a759-35de1be63004","Type":"ContainerStarted","Data":"6d7b95037ed66cda64ba090f8934b09f466a05780f4dd133c28ae0bdcd848432"} May 28 03:26:10.528243 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.528209 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6nvgb\" (UniqueName: \"kubernetes.io/projected/993a9ccc-e9ed-402b-b76f-92a749da0996-kube-api-access-6nvgb\") pod \"kuadrant-operator-catalog-rlfk4\" (UID: \"993a9ccc-e9ed-402b-b76f-92a749da0996\") " pod="kuadrant-system/kuadrant-operator-catalog-rlfk4" May 28 03:26:10.537361 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.537325 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nvgb\" (UniqueName: \"kubernetes.io/projected/993a9ccc-e9ed-402b-b76f-92a749da0996-kube-api-access-6nvgb\") pod \"kuadrant-operator-catalog-rlfk4\" (UID: \"993a9ccc-e9ed-402b-b76f-92a749da0996\") " pod="kuadrant-system/kuadrant-operator-catalog-rlfk4" May 28 03:26:10.685175 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.685084 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rlfk4" May 28 03:26:10.820481 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:10.820455 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rlfk4"] May 28 03:26:10.831436 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:26:10.831407 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod993a9ccc_e9ed_402b_b76f_92a749da0996.slice/crio-a00e521f8eb87c28dd25f5acad2dbbc050b3b9f8def147d96d1415695b8a5357 WatchSource:0}: Error finding container a00e521f8eb87c28dd25f5acad2dbbc050b3b9f8def147d96d1415695b8a5357: Status 404 returned error can't find the container with id a00e521f8eb87c28dd25f5acad2dbbc050b3b9f8def147d96d1415695b8a5357 May 28 03:26:11.517185 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.517121 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rlfk4" event={"ID":"993a9ccc-e9ed-402b-b76f-92a749da0996","Type":"ContainerStarted","Data":"a00e521f8eb87c28dd25f5acad2dbbc050b3b9f8def147d96d1415695b8a5357"} May 28 03:26:11.816574 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.816485 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs"] May 28 03:26:11.819369 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.819353 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:11.823258 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.823231 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wbw48\"" May 28 03:26:11.823668 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.823650 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" May 28 03:26:11.823887 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.823871 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" May 28 03:26:11.823951 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.823911 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" May 28 03:26:11.824019 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.824001 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" May 28 03:26:11.844524 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.844484 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs"] May 28 03:26:11.945030 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.944992 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/693c2353-4c65-4fac-bc7a-89aca2ae2f41-local-certs\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:11.945214 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.945043 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/693c2353-4c65-4fac-bc7a-89aca2ae2f41-cacerts\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:11.945214 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.945154 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqvg7\" (UniqueName: \"kubernetes.io/projected/693c2353-4c65-4fac-bc7a-89aca2ae2f41-kube-api-access-pqvg7\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:11.945214 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.945201 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/693c2353-4c65-4fac-bc7a-89aca2ae2f41-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:11.945387 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.945267 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/693c2353-4c65-4fac-bc7a-89aca2ae2f41-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:11.945387 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.945300 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693c2353-4c65-4fac-bc7a-89aca2ae2f41-istio-token\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:11.945387 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:11.945328 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/693c2353-4c65-4fac-bc7a-89aca2ae2f41-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.045705 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.045675 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pqvg7\" (UniqueName: \"kubernetes.io/projected/693c2353-4c65-4fac-bc7a-89aca2ae2f41-kube-api-access-pqvg7\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.045930 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.045720 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/693c2353-4c65-4fac-bc7a-89aca2ae2f41-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.045930 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.045779 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/693c2353-4c65-4fac-bc7a-89aca2ae2f41-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.045930 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.045809 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693c2353-4c65-4fac-bc7a-89aca2ae2f41-istio-token\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.045930 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.045836 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/693c2353-4c65-4fac-bc7a-89aca2ae2f41-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.045930 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.045903 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/693c2353-4c65-4fac-bc7a-89aca2ae2f41-local-certs\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.046248 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.045954 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/693c2353-4c65-4fac-bc7a-89aca2ae2f41-cacerts\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.046440 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.046415 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/693c2353-4c65-4fac-bc7a-89aca2ae2f41-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.048882 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.048817 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/693c2353-4c65-4fac-bc7a-89aca2ae2f41-local-certs\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.049011 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.048932 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/693c2353-4c65-4fac-bc7a-89aca2ae2f41-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.049271 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.049249 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/693c2353-4c65-4fac-bc7a-89aca2ae2f41-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.049683 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.049662 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/693c2353-4c65-4fac-bc7a-89aca2ae2f41-cacerts\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.062893 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.062865 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/693c2353-4c65-4fac-bc7a-89aca2ae2f41-istio-token\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.063033 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.062969 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqvg7\" (UniqueName: \"kubernetes.io/projected/693c2353-4c65-4fac-bc7a-89aca2ae2f41-kube-api-access-pqvg7\") pod \"istiod-openshift-gateway-798958bb55-9hdvs\" (UID: \"693c2353-4c65-4fac-bc7a-89aca2ae2f41\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.128894 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.128800 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:12.665266 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:12.665232 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs"] May 28 03:26:12.666427 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:26:12.666388 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod693c2353_4c65_4fac_bc7a_89aca2ae2f41.slice/crio-55d6585bac5e1a4d5346f893899e0b8986cdffbef592a48eee64aef2e159ab31 WatchSource:0}: Error finding container 55d6585bac5e1a4d5346f893899e0b8986cdffbef592a48eee64aef2e159ab31: Status 404 returned error can't find the container with id 55d6585bac5e1a4d5346f893899e0b8986cdffbef592a48eee64aef2e159ab31 May 28 03:26:13.527107 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:13.527056 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" event={"ID":"693c2353-4c65-4fac-bc7a-89aca2ae2f41","Type":"ContainerStarted","Data":"55d6585bac5e1a4d5346f893899e0b8986cdffbef592a48eee64aef2e159ab31"} May 28 03:26:13.529317 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:13.529276 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mmjwk" event={"ID":"1e93a8c6-fa42-4bb7-a759-35de1be63004","Type":"ContainerStarted","Data":"0aade01152fd9f4dab1f331a371a7d75fb73dfd361fa368ca2f1077eb22bd141"} May 28 03:26:13.529477 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:13.529408 2555 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-mmjwk" podUID="1e93a8c6-fa42-4bb7-a759-35de1be63004" containerName="registry-server" containerID="cri-o://0aade01152fd9f4dab1f331a371a7d75fb73dfd361fa368ca2f1077eb22bd141" gracePeriod=2 May 28 03:26:13.530899 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:13.530867 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rlfk4" event={"ID":"993a9ccc-e9ed-402b-b76f-92a749da0996","Type":"ContainerStarted","Data":"fbc91711d1b50ad7f881758c6fa8d304b5693804a87c924e304314e2ae4ab475"} May 28 03:26:13.548338 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:13.547775 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-mmjwk" podStartSLOduration=2.273471378 podStartE2EDuration="4.547755205s" podCreationTimestamp="2026-05-28 03:26:09 +0000 UTC" firstStartedPulling="2026-05-28 03:26:10.281150221 +0000 UTC m=+562.991796177" lastFinishedPulling="2026-05-28 03:26:12.55543406 +0000 UTC m=+565.266080004" observedRunningTime="2026-05-28 03:26:13.546922999 +0000 UTC m=+566.257568957" watchObservedRunningTime="2026-05-28 03:26:13.547755205 +0000 UTC m=+566.258401168" May 28 03:26:13.567919 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:13.567678 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-rlfk4" podStartSLOduration=1.8416963069999999 podStartE2EDuration="3.567658006s" podCreationTimestamp="2026-05-28 03:26:10 +0000 UTC" firstStartedPulling="2026-05-28 03:26:10.832756713 +0000 UTC m=+563.543402653" lastFinishedPulling="2026-05-28 03:26:12.55871841 +0000 UTC m=+565.269364352" observedRunningTime="2026-05-28 03:26:13.567016951 +0000 UTC m=+566.277662917" watchObservedRunningTime="2026-05-28 03:26:13.567658006 +0000 UTC m=+566.278303992" May 28 03:26:13.808714 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:13.808685 2555 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmjwk" May 28 03:26:13.865380 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:13.865346 2555 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jjcl\" (UniqueName: \"kubernetes.io/projected/1e93a8c6-fa42-4bb7-a759-35de1be63004-kube-api-access-7jjcl\") pod \"1e93a8c6-fa42-4bb7-a759-35de1be63004\" (UID: \"1e93a8c6-fa42-4bb7-a759-35de1be63004\") " May 28 03:26:13.867892 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:13.867854 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e93a8c6-fa42-4bb7-a759-35de1be63004-kube-api-access-7jjcl" (OuterVolumeSpecName: "kube-api-access-7jjcl") pod "1e93a8c6-fa42-4bb7-a759-35de1be63004" (UID: "1e93a8c6-fa42-4bb7-a759-35de1be63004"). InnerVolumeSpecName "kube-api-access-7jjcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 03:26:13.966382 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:13.966341 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jjcl\" (UniqueName: \"kubernetes.io/projected/1e93a8c6-fa42-4bb7-a759-35de1be63004-kube-api-access-7jjcl\") on node \"ip-10-0-141-36.ec2.internal\" DevicePath \"\"" May 28 03:26:14.536302 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:14.536271 2555 generic.go:358] "Generic (PLEG): container finished" podID="1e93a8c6-fa42-4bb7-a759-35de1be63004" containerID="0aade01152fd9f4dab1f331a371a7d75fb73dfd361fa368ca2f1077eb22bd141" exitCode=0 May 28 03:26:14.536795 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:14.536340 2555 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmjwk" May 28 03:26:14.536795 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:14.536354 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mmjwk" event={"ID":"1e93a8c6-fa42-4bb7-a759-35de1be63004","Type":"ContainerDied","Data":"0aade01152fd9f4dab1f331a371a7d75fb73dfd361fa368ca2f1077eb22bd141"} May 28 03:26:14.536795 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:14.536402 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mmjwk" event={"ID":"1e93a8c6-fa42-4bb7-a759-35de1be63004","Type":"ContainerDied","Data":"6d7b95037ed66cda64ba090f8934b09f466a05780f4dd133c28ae0bdcd848432"} May 28 03:26:14.536795 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:14.536430 2555 scope.go:117] "RemoveContainer" containerID="0aade01152fd9f4dab1f331a371a7d75fb73dfd361fa368ca2f1077eb22bd141" May 28 03:26:14.553835 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:14.553807 2555 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmjwk"] May 28 03:26:14.558738 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:14.558713 2555 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmjwk"] May 28 03:26:15.204098 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:15.204068 2555 scope.go:117] "RemoveContainer" containerID="0aade01152fd9f4dab1f331a371a7d75fb73dfd361fa368ca2f1077eb22bd141" May 28 03:26:15.204427 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:26:15.204403 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0aade01152fd9f4dab1f331a371a7d75fb73dfd361fa368ca2f1077eb22bd141\": container with ID starting with 0aade01152fd9f4dab1f331a371a7d75fb73dfd361fa368ca2f1077eb22bd141 not found: ID does not exist" containerID="0aade01152fd9f4dab1f331a371a7d75fb73dfd361fa368ca2f1077eb22bd141" May 28 03:26:15.204530 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:15.204441 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0aade01152fd9f4dab1f331a371a7d75fb73dfd361fa368ca2f1077eb22bd141"} err="failed to get container status \"0aade01152fd9f4dab1f331a371a7d75fb73dfd361fa368ca2f1077eb22bd141\": rpc error: code = NotFound desc = could not find container \"0aade01152fd9f4dab1f331a371a7d75fb73dfd361fa368ca2f1077eb22bd141\": container with ID starting with 0aade01152fd9f4dab1f331a371a7d75fb73dfd361fa368ca2f1077eb22bd141 not found: ID does not exist" May 28 03:26:15.267104 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:15.267066 2555 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} May 28 03:26:15.267195 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:15.267138 2555 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} May 28 03:26:15.542170 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:15.542073 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" event={"ID":"693c2353-4c65-4fac-bc7a-89aca2ae2f41","Type":"ContainerStarted","Data":"4a8c1a5f885e07d0df8b7913fb46b2a7529581e80152f2efab388934b9208571"} May 28 03:26:15.542586 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:15.542197 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:15.571936 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:15.571880 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" podStartSLOduration=1.973494333 podStartE2EDuration="4.571863897s" podCreationTimestamp="2026-05-28 03:26:11 +0000 UTC" firstStartedPulling="2026-05-28 03:26:12.668460297 +0000 UTC m=+565.379106237" lastFinishedPulling="2026-05-28 03:26:15.266829861 +0000 UTC m=+567.977475801" observedRunningTime="2026-05-28 03:26:15.569469002 +0000 UTC m=+568.280114979" watchObservedRunningTime="2026-05-28 03:26:15.571863897 +0000 UTC m=+568.282509858" May 28 03:26:15.879381 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:15.879297 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e93a8c6-fa42-4bb7-a759-35de1be63004" path="/var/lib/kubelet/pods/1e93a8c6-fa42-4bb7-a759-35de1be63004/volumes" May 28 03:26:16.550120 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:16.550088 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9hdvs" May 28 03:26:20.685759 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:20.685718 2555 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-rlfk4" May 28 03:26:20.686166 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:20.685803 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-rlfk4" May 28 03:26:20.707305 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:20.707275 2555 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-rlfk4" May 28 03:26:21.585078 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:21.585053 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-rlfk4" May 28 03:26:36.881992 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:36.881959 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-tnn8l"] May 28 03:26:36.882438 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:36.882226 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e93a8c6-fa42-4bb7-a759-35de1be63004" containerName="registry-server" May 28 03:26:36.882438 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:36.882238 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e93a8c6-fa42-4bb7-a759-35de1be63004" containerName="registry-server" May 28 03:26:36.882438 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:36.882294 2555 memory_manager.go:356] "RemoveStaleState removing state" podUID="1e93a8c6-fa42-4bb7-a759-35de1be63004" containerName="registry-server" May 28 03:26:36.886221 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:36.886204 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-tnn8l" May 28 03:26:36.889668 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:36.889646 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bt5vq\"" May 28 03:26:36.901474 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:36.901447 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-tnn8l"] May 28 03:26:37.066380 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:37.066343 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdj7c\" (UniqueName: \"kubernetes.io/projected/9733288e-7ee5-4015-b1a0-ec6c948dc7b4-kube-api-access-wdj7c\") pod \"authorino-operator-657f44b778-tnn8l\" (UID: \"9733288e-7ee5-4015-b1a0-ec6c948dc7b4\") " pod="kuadrant-system/authorino-operator-657f44b778-tnn8l" May 28 03:26:37.166744 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:37.166711 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wdj7c\" (UniqueName: \"kubernetes.io/projected/9733288e-7ee5-4015-b1a0-ec6c948dc7b4-kube-api-access-wdj7c\") pod \"authorino-operator-657f44b778-tnn8l\" (UID: \"9733288e-7ee5-4015-b1a0-ec6c948dc7b4\") " pod="kuadrant-system/authorino-operator-657f44b778-tnn8l" May 28 03:26:37.180388 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:37.180358 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdj7c\" (UniqueName: \"kubernetes.io/projected/9733288e-7ee5-4015-b1a0-ec6c948dc7b4-kube-api-access-wdj7c\") pod \"authorino-operator-657f44b778-tnn8l\" (UID: \"9733288e-7ee5-4015-b1a0-ec6c948dc7b4\") " pod="kuadrant-system/authorino-operator-657f44b778-tnn8l" May 28 03:26:37.196318 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:37.196284 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-tnn8l" May 28 03:26:37.332543 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:37.332506 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-tnn8l"] May 28 03:26:37.336149 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:26:37.336113 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9733288e_7ee5_4015_b1a0_ec6c948dc7b4.slice/crio-358527f0e28d0acba4c172dcd08477b1efab13b7d220f78292aacae11e78c02f WatchSource:0}: Error finding container 358527f0e28d0acba4c172dcd08477b1efab13b7d220f78292aacae11e78c02f: Status 404 returned error can't find the container with id 358527f0e28d0acba4c172dcd08477b1efab13b7d220f78292aacae11e78c02f May 28 03:26:37.613850 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:37.613763 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-tnn8l" event={"ID":"9733288e-7ee5-4015-b1a0-ec6c948dc7b4","Type":"ContainerStarted","Data":"358527f0e28d0acba4c172dcd08477b1efab13b7d220f78292aacae11e78c02f"} May 28 03:26:39.621796 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:39.621703 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-tnn8l" event={"ID":"9733288e-7ee5-4015-b1a0-ec6c948dc7b4","Type":"ContainerStarted","Data":"33f8a13f94add19534b81e452aa3f5c9e6036efa00d5c4a1cd66f4d8420de8b0"} May 28 03:26:39.621796 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:39.621768 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-tnn8l" May 28 03:26:39.642464 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:39.642411 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-tnn8l" podStartSLOduration=1.649981497 podStartE2EDuration="3.642396414s" podCreationTimestamp="2026-05-28 03:26:36 +0000 UTC" firstStartedPulling="2026-05-28 03:26:37.338620376 +0000 UTC m=+590.049266317" lastFinishedPulling="2026-05-28 03:26:39.331035281 +0000 UTC m=+592.041681234" observedRunningTime="2026-05-28 03:26:39.639974802 +0000 UTC m=+592.350620773" watchObservedRunningTime="2026-05-28 03:26:39.642396414 +0000 UTC m=+592.353042375" May 28 03:26:47.783109 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:47.783080 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:26:47.783616 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:47.783453 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:26:50.626825 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:26:50.626793 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-tnn8l" May 28 03:27:03.617098 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:03.617059 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bvn4k"] May 28 03:27:03.624144 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:03.624114 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bvn4k" May 28 03:27:03.626839 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:03.626736 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-x88fg\"" May 28 03:27:03.632128 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:03.631772 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bvn4k"] May 28 03:27:03.677871 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:03.677824 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn4mj\" (UniqueName: \"kubernetes.io/projected/2a3c1b6d-a16c-4c2a-85e3-0d07912e5892-kube-api-access-cn4mj\") pod \"limitador-operator-controller-manager-85c4996f8c-bvn4k\" (UID: \"2a3c1b6d-a16c-4c2a-85e3-0d07912e5892\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bvn4k" May 28 03:27:03.779049 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:03.779008 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cn4mj\" (UniqueName: \"kubernetes.io/projected/2a3c1b6d-a16c-4c2a-85e3-0d07912e5892-kube-api-access-cn4mj\") pod \"limitador-operator-controller-manager-85c4996f8c-bvn4k\" (UID: \"2a3c1b6d-a16c-4c2a-85e3-0d07912e5892\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bvn4k" May 28 03:27:03.791319 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:03.791283 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn4mj\" (UniqueName: \"kubernetes.io/projected/2a3c1b6d-a16c-4c2a-85e3-0d07912e5892-kube-api-access-cn4mj\") pod \"limitador-operator-controller-manager-85c4996f8c-bvn4k\" (UID: \"2a3c1b6d-a16c-4c2a-85e3-0d07912e5892\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bvn4k" May 28 03:27:03.935806 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:03.935770 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bvn4k" May 28 03:27:04.077232 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:04.077197 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bvn4k"] May 28 03:27:04.080860 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:27:04.080828 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3c1b6d_a16c_4c2a_85e3_0d07912e5892.slice/crio-53215da04080dc4486feb283b3285c5da5da1e33cd6a8684a355c5b7e7a18db6 WatchSource:0}: Error finding container 53215da04080dc4486feb283b3285c5da5da1e33cd6a8684a355c5b7e7a18db6: Status 404 returned error can't find the container with id 53215da04080dc4486feb283b3285c5da5da1e33cd6a8684a355c5b7e7a18db6 May 28 03:27:04.703637 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:04.703592 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bvn4k" event={"ID":"2a3c1b6d-a16c-4c2a-85e3-0d07912e5892","Type":"ContainerStarted","Data":"53215da04080dc4486feb283b3285c5da5da1e33cd6a8684a355c5b7e7a18db6"} May 28 03:27:05.709707 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:05.709673 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bvn4k" event={"ID":"2a3c1b6d-a16c-4c2a-85e3-0d07912e5892","Type":"ContainerStarted","Data":"63e61ea1b070b911dccf0df2fb479d8b7872540f0ed733770f6f5b105900e37b"} May 28 03:27:05.710089 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:05.709805 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bvn4k" May 28 03:27:05.733944 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:05.733885 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bvn4k" podStartSLOduration=1.520774074 podStartE2EDuration="2.733868497s" podCreationTimestamp="2026-05-28 03:27:03 +0000 UTC" firstStartedPulling="2026-05-28 03:27:04.083113496 +0000 UTC m=+616.793759436" lastFinishedPulling="2026-05-28 03:27:05.296207916 +0000 UTC m=+618.006853859" observedRunningTime="2026-05-28 03:27:05.73237803 +0000 UTC m=+618.443024005" watchObservedRunningTime="2026-05-28 03:27:05.733868497 +0000 UTC m=+618.444514458" May 28 03:27:16.715179 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:27:16.715096 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-bvn4k" May 28 03:28:13.534818 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.534782 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b59dcc5cb-wsn95"] May 28 03:28:13.537536 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.537506 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b59dcc5cb-wsn95" May 28 03:28:13.539984 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.539960 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" May 28 03:28:13.540508 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.540476 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m26n2\"" May 28 03:28:13.560784 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.560735 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b59dcc5cb-wsn95"] May 28 03:28:13.644095 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.644034 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsn4m\" (UniqueName: \"kubernetes.io/projected/6b15d088-57e2-4319-b6b1-a0b7d66206d4-kube-api-access-wsn4m\") pod \"maas-controller-b59dcc5cb-wsn95\" (UID: \"6b15d088-57e2-4319-b6b1-a0b7d66206d4\") " pod="opendatahub/maas-controller-b59dcc5cb-wsn95" May 28 03:28:13.697080 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.697036 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84d8b48b59-vbcqc"] May 28 03:28:13.699451 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.699435 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d8b48b59-vbcqc" May 28 03:28:13.722196 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.722158 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d8b48b59-vbcqc"] May 28 03:28:13.744955 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.744905 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45qv7\" (UniqueName: \"kubernetes.io/projected/be48d37b-56e5-41e2-b72f-503f02226b9e-kube-api-access-45qv7\") pod \"maas-controller-84d8b48b59-vbcqc\" (UID: \"be48d37b-56e5-41e2-b72f-503f02226b9e\") " pod="opendatahub/maas-controller-84d8b48b59-vbcqc" May 28 03:28:13.745142 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.744979 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wsn4m\" (UniqueName: \"kubernetes.io/projected/6b15d088-57e2-4319-b6b1-a0b7d66206d4-kube-api-access-wsn4m\") pod \"maas-controller-b59dcc5cb-wsn95\" (UID: \"6b15d088-57e2-4319-b6b1-a0b7d66206d4\") " pod="opendatahub/maas-controller-b59dcc5cb-wsn95" May 28 03:28:13.754361 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.754325 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsn4m\" (UniqueName: \"kubernetes.io/projected/6b15d088-57e2-4319-b6b1-a0b7d66206d4-kube-api-access-wsn4m\") pod \"maas-controller-b59dcc5cb-wsn95\" (UID: \"6b15d088-57e2-4319-b6b1-a0b7d66206d4\") " pod="opendatahub/maas-controller-b59dcc5cb-wsn95" May 28 03:28:13.846337 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.846239 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-45qv7\" (UniqueName: \"kubernetes.io/projected/be48d37b-56e5-41e2-b72f-503f02226b9e-kube-api-access-45qv7\") pod \"maas-controller-84d8b48b59-vbcqc\" (UID: \"be48d37b-56e5-41e2-b72f-503f02226b9e\") " pod="opendatahub/maas-controller-84d8b48b59-vbcqc" May 28 03:28:13.846651 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.846635 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b59dcc5cb-wsn95" May 28 03:28:13.854694 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.854661 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45qv7\" (UniqueName: \"kubernetes.io/projected/be48d37b-56e5-41e2-b72f-503f02226b9e-kube-api-access-45qv7\") pod \"maas-controller-84d8b48b59-vbcqc\" (UID: \"be48d37b-56e5-41e2-b72f-503f02226b9e\") " pod="opendatahub/maas-controller-84d8b48b59-vbcqc" May 28 03:28:13.998744 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:13.998678 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b59dcc5cb-wsn95"] May 28 03:28:14.001692 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:28:14.001658 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b15d088_57e2_4319_b6b1_a0b7d66206d4.slice/crio-f66bd639c05a47cdaa68ed787479feb2fb511b6d226076c1634d0127d93f8bc5 WatchSource:0}: Error finding container f66bd639c05a47cdaa68ed787479feb2fb511b6d226076c1634d0127d93f8bc5: Status 404 returned error can't find the container with id f66bd639c05a47cdaa68ed787479feb2fb511b6d226076c1634d0127d93f8bc5 May 28 03:28:14.009532 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:14.009503 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d8b48b59-vbcqc" May 28 03:28:14.141365 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:14.141283 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d8b48b59-vbcqc"] May 28 03:28:14.144410 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:28:14.144373 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe48d37b_56e5_41e2_b72f_503f02226b9e.slice/crio-774218a5bacdce18d0d44acd6e858178cd2ba5e54b37a55340d34b16a138f0a2 WatchSource:0}: Error finding container 774218a5bacdce18d0d44acd6e858178cd2ba5e54b37a55340d34b16a138f0a2: Status 404 returned error can't find the container with id 774218a5bacdce18d0d44acd6e858178cd2ba5e54b37a55340d34b16a138f0a2 May 28 03:28:14.944888 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:14.944828 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d8b48b59-vbcqc" event={"ID":"be48d37b-56e5-41e2-b72f-503f02226b9e","Type":"ContainerStarted","Data":"774218a5bacdce18d0d44acd6e858178cd2ba5e54b37a55340d34b16a138f0a2"} May 28 03:28:14.946372 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:14.946336 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b59dcc5cb-wsn95" event={"ID":"6b15d088-57e2-4319-b6b1-a0b7d66206d4","Type":"ContainerStarted","Data":"f66bd639c05a47cdaa68ed787479feb2fb511b6d226076c1634d0127d93f8bc5"} May 28 03:28:16.955631 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:16.955587 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d8b48b59-vbcqc" event={"ID":"be48d37b-56e5-41e2-b72f-503f02226b9e","Type":"ContainerStarted","Data":"a1f9c7714512134872d77294a1151727e97cb7b4fefff00c7207634a8c3682e9"} May 28 03:28:16.955631 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:16.955634 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84d8b48b59-vbcqc" May 28 03:28:16.956885 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:16.956859 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b59dcc5cb-wsn95" event={"ID":"6b15d088-57e2-4319-b6b1-a0b7d66206d4","Type":"ContainerStarted","Data":"aa33fa985e07e4c32aedeb239e048d37649f0ece4261887180fa6dd397c9811a"} May 28 03:28:16.957002 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:16.956969 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b59dcc5cb-wsn95" May 28 03:28:16.974686 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:16.974630 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-84d8b48b59-vbcqc" podStartSLOduration=1.363464527 podStartE2EDuration="3.974613845s" podCreationTimestamp="2026-05-28 03:28:13 +0000 UTC" firstStartedPulling="2026-05-28 03:28:14.145793191 +0000 UTC m=+686.856439131" lastFinishedPulling="2026-05-28 03:28:16.756942506 +0000 UTC m=+689.467588449" observedRunningTime="2026-05-28 03:28:16.972717687 +0000 UTC m=+689.683363649" watchObservedRunningTime="2026-05-28 03:28:16.974613845 +0000 UTC m=+689.685259871" May 28 03:28:16.992635 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:16.992574 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-b59dcc5cb-wsn95" podStartSLOduration=1.243961259 podStartE2EDuration="3.992551741s" podCreationTimestamp="2026-05-28 03:28:13 +0000 UTC" firstStartedPulling="2026-05-28 03:28:14.003296707 +0000 UTC m=+686.713942647" lastFinishedPulling="2026-05-28 03:28:16.751887188 +0000 UTC m=+689.462533129" observedRunningTime="2026-05-28 03:28:16.991000136 +0000 UTC m=+689.701646122" watchObservedRunningTime="2026-05-28 03:28:16.992551741 +0000 UTC m=+689.703197706" May 28 03:28:27.965355 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:27.965323 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b59dcc5cb-wsn95" May 28 03:28:27.983737 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:27.983700 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84d8b48b59-vbcqc" May 28 03:28:28.078508 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:28.078453 2555 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b59dcc5cb-wsn95"] May 28 03:28:28.078769 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:28.078741 2555 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-b59dcc5cb-wsn95" podUID="6b15d088-57e2-4319-b6b1-a0b7d66206d4" containerName="manager" containerID="cri-o://aa33fa985e07e4c32aedeb239e048d37649f0ece4261887180fa6dd397c9811a" gracePeriod=30 May 28 03:28:28.319354 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:28.319329 2555 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b59dcc5cb-wsn95" May 28 03:28:28.377194 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:28.377154 2555 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsn4m\" (UniqueName: \"kubernetes.io/projected/6b15d088-57e2-4319-b6b1-a0b7d66206d4-kube-api-access-wsn4m\") pod \"6b15d088-57e2-4319-b6b1-a0b7d66206d4\" (UID: \"6b15d088-57e2-4319-b6b1-a0b7d66206d4\") " May 28 03:28:28.379624 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:28.379593 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b15d088-57e2-4319-b6b1-a0b7d66206d4-kube-api-access-wsn4m" (OuterVolumeSpecName: "kube-api-access-wsn4m") pod "6b15d088-57e2-4319-b6b1-a0b7d66206d4" (UID: "6b15d088-57e2-4319-b6b1-a0b7d66206d4"). InnerVolumeSpecName "kube-api-access-wsn4m". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 03:28:28.478237 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:28.478197 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsn4m\" (UniqueName: \"kubernetes.io/projected/6b15d088-57e2-4319-b6b1-a0b7d66206d4-kube-api-access-wsn4m\") on node \"ip-10-0-141-36.ec2.internal\" DevicePath \"\"" May 28 03:28:28.999758 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:28.999724 2555 generic.go:358] "Generic (PLEG): container finished" podID="6b15d088-57e2-4319-b6b1-a0b7d66206d4" containerID="aa33fa985e07e4c32aedeb239e048d37649f0ece4261887180fa6dd397c9811a" exitCode=0 May 28 03:28:29.000171 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:28.999786 2555 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b59dcc5cb-wsn95" May 28 03:28:29.000171 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:28.999811 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b59dcc5cb-wsn95" event={"ID":"6b15d088-57e2-4319-b6b1-a0b7d66206d4","Type":"ContainerDied","Data":"aa33fa985e07e4c32aedeb239e048d37649f0ece4261887180fa6dd397c9811a"} May 28 03:28:29.000171 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:28.999849 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b59dcc5cb-wsn95" event={"ID":"6b15d088-57e2-4319-b6b1-a0b7d66206d4","Type":"ContainerDied","Data":"f66bd639c05a47cdaa68ed787479feb2fb511b6d226076c1634d0127d93f8bc5"} May 28 03:28:29.000171 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:28.999867 2555 scope.go:117] "RemoveContainer" containerID="aa33fa985e07e4c32aedeb239e048d37649f0ece4261887180fa6dd397c9811a" May 28 03:28:29.008521 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:29.008479 2555 scope.go:117] "RemoveContainer" containerID="aa33fa985e07e4c32aedeb239e048d37649f0ece4261887180fa6dd397c9811a" May 28 03:28:29.008837 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:28:29.008815 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa33fa985e07e4c32aedeb239e048d37649f0ece4261887180fa6dd397c9811a\": container with ID starting with aa33fa985e07e4c32aedeb239e048d37649f0ece4261887180fa6dd397c9811a not found: ID does not exist" containerID="aa33fa985e07e4c32aedeb239e048d37649f0ece4261887180fa6dd397c9811a" May 28 03:28:29.008919 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:29.008850 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa33fa985e07e4c32aedeb239e048d37649f0ece4261887180fa6dd397c9811a"} err="failed to get container status \"aa33fa985e07e4c32aedeb239e048d37649f0ece4261887180fa6dd397c9811a\": rpc error: code = NotFound desc = could not find container \"aa33fa985e07e4c32aedeb239e048d37649f0ece4261887180fa6dd397c9811a\": container with ID starting with aa33fa985e07e4c32aedeb239e048d37649f0ece4261887180fa6dd397c9811a not found: ID does not exist" May 28 03:28:29.023587 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:29.023548 2555 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b59dcc5cb-wsn95"] May 28 03:28:29.028004 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:29.027973 2555 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b59dcc5cb-wsn95"] May 28 03:28:29.878452 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:29.878421 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b15d088-57e2-4319-b6b1-a0b7d66206d4" path="/var/lib/kubelet/pods/6b15d088-57e2-4319-b6b1-a0b7d66206d4/volumes" May 28 03:28:48.413734 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.413691 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6f94d9d45c-bmpnb"] May 28 03:28:48.414195 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.414004 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b15d088-57e2-4319-b6b1-a0b7d66206d4" containerName="manager" May 28 03:28:48.414195 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.414014 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b15d088-57e2-4319-b6b1-a0b7d66206d4" containerName="manager" May 28 03:28:48.414195 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.414058 2555 memory_manager.go:356] "RemoveStaleState removing state" podUID="6b15d088-57e2-4319-b6b1-a0b7d66206d4" containerName="manager" May 28 03:28:48.416353 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.416333 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6f94d9d45c-bmpnb" May 28 03:28:48.419829 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.419809 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" May 28 03:28:48.419909 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.419812 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-lng4z\"" May 28 03:28:48.429612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.429585 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6f94d9d45c-bmpnb"] May 28 03:28:48.548777 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.548736 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6c40d20-0624-4dae-96d0-e4ef7169a39b-maas-api-tls\") pod \"maas-api-6f94d9d45c-bmpnb\" (UID: \"b6c40d20-0624-4dae-96d0-e4ef7169a39b\") " pod="opendatahub/maas-api-6f94d9d45c-bmpnb" May 28 03:28:48.548963 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.548810 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkm6v\" (UniqueName: \"kubernetes.io/projected/b6c40d20-0624-4dae-96d0-e4ef7169a39b-kube-api-access-xkm6v\") pod \"maas-api-6f94d9d45c-bmpnb\" (UID: \"b6c40d20-0624-4dae-96d0-e4ef7169a39b\") " pod="opendatahub/maas-api-6f94d9d45c-bmpnb" May 28 03:28:48.649706 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.649666 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xkm6v\" (UniqueName: \"kubernetes.io/projected/b6c40d20-0624-4dae-96d0-e4ef7169a39b-kube-api-access-xkm6v\") pod \"maas-api-6f94d9d45c-bmpnb\" (UID: \"b6c40d20-0624-4dae-96d0-e4ef7169a39b\") " pod="opendatahub/maas-api-6f94d9d45c-bmpnb" May 28 03:28:48.649902 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.649734 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6c40d20-0624-4dae-96d0-e4ef7169a39b-maas-api-tls\") pod \"maas-api-6f94d9d45c-bmpnb\" (UID: \"b6c40d20-0624-4dae-96d0-e4ef7169a39b\") " pod="opendatahub/maas-api-6f94d9d45c-bmpnb" May 28 03:28:48.652240 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.652212 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6c40d20-0624-4dae-96d0-e4ef7169a39b-maas-api-tls\") pod \"maas-api-6f94d9d45c-bmpnb\" (UID: \"b6c40d20-0624-4dae-96d0-e4ef7169a39b\") " pod="opendatahub/maas-api-6f94d9d45c-bmpnb" May 28 03:28:48.658069 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.658038 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkm6v\" (UniqueName: \"kubernetes.io/projected/b6c40d20-0624-4dae-96d0-e4ef7169a39b-kube-api-access-xkm6v\") pod \"maas-api-6f94d9d45c-bmpnb\" (UID: \"b6c40d20-0624-4dae-96d0-e4ef7169a39b\") " pod="opendatahub/maas-api-6f94d9d45c-bmpnb" May 28 03:28:48.726795 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.726690 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6f94d9d45c-bmpnb" May 28 03:28:48.792451 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.792216 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-hvdwr"] May 28 03:28:48.795912 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.795882 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-hvdwr" May 28 03:28:48.798742 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.798681 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lhdrv\"" May 28 03:28:48.798923 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.798789 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" May 28 03:28:48.804830 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.804784 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-hvdwr"] May 28 03:28:48.878453 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.878274 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6f94d9d45c-bmpnb"] May 28 03:28:48.881253 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:28:48.881213 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6c40d20_0624_4dae_96d0_e4ef7169a39b.slice/crio-b66ea3707fb61582a91581022d49f436720ee1e4aa9a2e6e48288161e9fe65aa WatchSource:0}: Error finding container b66ea3707fb61582a91581022d49f436720ee1e4aa9a2e6e48288161e9fe65aa: Status 404 returned error can't find the container with id b66ea3707fb61582a91581022d49f436720ee1e4aa9a2e6e48288161e9fe65aa May 28 03:28:48.951445 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:48.951409 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwcq8\" (UniqueName: \"kubernetes.io/projected/65d3c4ce-2c21-4961-9472-a822c5e43d52-kube-api-access-kwcq8\") pod \"payload-processing-58b6c8fdc7-hvdwr\" (UID: \"65d3c4ce-2c21-4961-9472-a822c5e43d52\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-hvdwr" May 28 03:28:49.052881 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:49.052782 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kwcq8\" (UniqueName: \"kubernetes.io/projected/65d3c4ce-2c21-4961-9472-a822c5e43d52-kube-api-access-kwcq8\") pod \"payload-processing-58b6c8fdc7-hvdwr\" (UID: \"65d3c4ce-2c21-4961-9472-a822c5e43d52\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-hvdwr" May 28 03:28:49.066662 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:49.066630 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwcq8\" (UniqueName: \"kubernetes.io/projected/65d3c4ce-2c21-4961-9472-a822c5e43d52-kube-api-access-kwcq8\") pod \"payload-processing-58b6c8fdc7-hvdwr\" (UID: \"65d3c4ce-2c21-4961-9472-a822c5e43d52\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-hvdwr" May 28 03:28:49.074714 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:49.074680 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6f94d9d45c-bmpnb" event={"ID":"b6c40d20-0624-4dae-96d0-e4ef7169a39b","Type":"ContainerStarted","Data":"b66ea3707fb61582a91581022d49f436720ee1e4aa9a2e6e48288161e9fe65aa"} May 28 03:28:49.111674 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:49.111637 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-hvdwr" May 28 03:28:49.241287 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:49.241109 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-hvdwr"] May 28 03:28:49.244115 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:28:49.244071 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d3c4ce_2c21_4961_9472_a822c5e43d52.slice/crio-25daa55acf622c81d8bbf244eebee7b234a31bf1b947ba027935e131d4fd84de WatchSource:0}: Error finding container 25daa55acf622c81d8bbf244eebee7b234a31bf1b947ba027935e131d4fd84de: Status 404 returned error can't find the container with id 25daa55acf622c81d8bbf244eebee7b234a31bf1b947ba027935e131d4fd84de May 28 03:28:50.079991 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:50.079939 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-hvdwr" event={"ID":"65d3c4ce-2c21-4961-9472-a822c5e43d52","Type":"ContainerStarted","Data":"25daa55acf622c81d8bbf244eebee7b234a31bf1b947ba027935e131d4fd84de"} May 28 03:28:51.085243 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:51.085210 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6f94d9d45c-bmpnb" event={"ID":"b6c40d20-0624-4dae-96d0-e4ef7169a39b","Type":"ContainerStarted","Data":"0d3a790ce1e76aaad0ccfc5e3069d95c9b0744470024fedcb881977ebda883c2"} May 28 03:28:51.085630 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:51.085273 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6f94d9d45c-bmpnb" May 28 03:28:51.086642 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:51.086618 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-hvdwr" event={"ID":"65d3c4ce-2c21-4961-9472-a822c5e43d52","Type":"ContainerStarted","Data":"61a930aad55ea8fc8cf5219106c8d797d3800ac44185ef6186aca056cae2a3ad"} May 28 03:28:51.086777 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:51.086764 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-hvdwr" May 28 03:28:51.105784 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:51.105726 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-6f94d9d45c-bmpnb" podStartSLOduration=1.046146567 podStartE2EDuration="3.105711322s" podCreationTimestamp="2026-05-28 03:28:48 +0000 UTC" firstStartedPulling="2026-05-28 03:28:48.882995478 +0000 UTC m=+721.593641418" lastFinishedPulling="2026-05-28 03:28:50.942560233 +0000 UTC m=+723.653206173" observedRunningTime="2026-05-28 03:28:51.103466948 +0000 UTC m=+723.814112924" watchObservedRunningTime="2026-05-28 03:28:51.105711322 +0000 UTC m=+723.816357284" May 28 03:28:51.120407 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:51.120351 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-hvdwr" podStartSLOduration=1.421531594 podStartE2EDuration="3.120334032s" podCreationTimestamp="2026-05-28 03:28:48 +0000 UTC" firstStartedPulling="2026-05-28 03:28:49.245960966 +0000 UTC m=+721.956606907" lastFinishedPulling="2026-05-28 03:28:50.944763392 +0000 UTC m=+723.655409345" observedRunningTime="2026-05-28 03:28:51.119118636 +0000 UTC m=+723.829764619" watchObservedRunningTime="2026-05-28 03:28:51.120334032 +0000 UTC m=+723.830979994" May 28 03:28:57.096415 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:28:57.096374 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6f94d9d45c-bmpnb" May 28 03:29:02.092679 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:02.092648 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-hvdwr" May 28 03:29:17.902763 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:17.902722 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c"] May 28 03:29:17.906308 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:17.906285 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:17.909572 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:17.909551 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" May 28 03:29:17.909572 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:17.909565 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" May 28 03:29:17.909773 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:17.909553 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" May 28 03:29:17.909773 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:17.909598 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9vdxf\"" May 28 03:29:17.916563 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:17.916536 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c"] May 28 03:29:17.985614 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:17.985579 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc66r\" (UniqueName: \"kubernetes.io/projected/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-kube-api-access-nc66r\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:17.985820 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:17.985636 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:17.985820 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:17.985717 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:17.985820 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:17.985747 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:17.985820 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:17.985769 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:17.985820 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:17.985795 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.087063 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.087013 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.087063 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.087060 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.087307 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.087222 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.087377 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.087316 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nc66r\" (UniqueName: \"kubernetes.io/projected/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-kube-api-access-nc66r\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.087443 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.087384 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.087510 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.087439 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.087510 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.087441 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.087749 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.087727 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.087847 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.087797 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.089610 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.089585 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.090162 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.090146 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.095321 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.095300 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc66r\" (UniqueName: \"kubernetes.io/projected/ff537ae8-ac1e-4c0c-87d9-47805ccfe77b-kube-api-access-nc66r\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-94v5c\" (UID: \"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.216355 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.216322 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:18.356066 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:18.355952 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c"] May 28 03:29:18.360572 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:29:18.360541 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff537ae8_ac1e_4c0c_87d9_47805ccfe77b.slice/crio-eb331545e807ec25d1029cfa277f794882ccc2103bb6211d29ba9e55a5467481 WatchSource:0}: Error finding container eb331545e807ec25d1029cfa277f794882ccc2103bb6211d29ba9e55a5467481: Status 404 returned error can't find the container with id eb331545e807ec25d1029cfa277f794882ccc2103bb6211d29ba9e55a5467481 May 28 03:29:19.189941 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:19.189899 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" event={"ID":"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b","Type":"ContainerStarted","Data":"eb331545e807ec25d1029cfa277f794882ccc2103bb6211d29ba9e55a5467481"} May 28 03:29:24.214182 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:24.214145 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" event={"ID":"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b","Type":"ContainerStarted","Data":"32bf5bf038e568606fc95f1c15439fbe96acfbf4d2273f801dc8e6c467e78cad"} May 28 03:29:33.249869 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:33.249828 2555 generic.go:358] "Generic (PLEG): container finished" podID="ff537ae8-ac1e-4c0c-87d9-47805ccfe77b" containerID="32bf5bf038e568606fc95f1c15439fbe96acfbf4d2273f801dc8e6c467e78cad" exitCode=0 May 28 03:29:33.250348 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:33.250014 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" event={"ID":"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b","Type":"ContainerDied","Data":"32bf5bf038e568606fc95f1c15439fbe96acfbf4d2273f801dc8e6c467e78cad"} May 28 03:29:33.251088 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:33.251065 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 03:29:35.260077 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:35.260036 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" event={"ID":"ff537ae8-ac1e-4c0c-87d9-47805ccfe77b","Type":"ContainerStarted","Data":"982b9c696705f00ff51595b742f6186aa4e8fb0d03d845fd9172ff765820d1af"} May 28 03:29:35.260468 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:35.260249 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:29:35.282179 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:35.282128 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" podStartSLOduration=2.299696283 podStartE2EDuration="18.282113616s" podCreationTimestamp="2026-05-28 03:29:17 +0000 UTC" firstStartedPulling="2026-05-28 03:29:18.362704832 +0000 UTC m=+751.073350773" lastFinishedPulling="2026-05-28 03:29:34.345122166 +0000 UTC m=+767.055768106" observedRunningTime="2026-05-28 03:29:35.279905101 +0000 UTC m=+767.990551065" watchObservedRunningTime="2026-05-28 03:29:35.282113616 +0000 UTC m=+767.992759577" May 28 03:29:46.276649 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:29:46.276616 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-94v5c" May 28 03:30:00.702274 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.702239 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw"] May 28 03:30:00.729611 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.729577 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw"] May 28 03:30:00.729770 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.729710 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.732271 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.732249 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" May 28 03:30:00.752539 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.752481 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrtcf\" (UniqueName: \"kubernetes.io/projected/a9a4727f-6d8b-4581-9f69-6a4694ecb433-kube-api-access-hrtcf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.752703 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.752555 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9a4727f-6d8b-4581-9f69-6a4694ecb433-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.752703 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.752606 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9a4727f-6d8b-4581-9f69-6a4694ecb433-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.752703 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.752647 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9a4727f-6d8b-4581-9f69-6a4694ecb433-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.752703 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.752697 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9a4727f-6d8b-4581-9f69-6a4694ecb433-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.752904 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.752721 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a9a4727f-6d8b-4581-9f69-6a4694ecb433-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.853271 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.853227 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9a4727f-6d8b-4581-9f69-6a4694ecb433-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.853990 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.853951 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9a4727f-6d8b-4581-9f69-6a4694ecb433-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.854405 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.854371 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9a4727f-6d8b-4581-9f69-6a4694ecb433-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.855021 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.854999 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9a4727f-6d8b-4581-9f69-6a4694ecb433-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.855161 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.854288 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9a4727f-6d8b-4581-9f69-6a4694ecb433-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.855322 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.855159 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a9a4727f-6d8b-4581-9f69-6a4694ecb433-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.855433 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.855410 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a9a4727f-6d8b-4581-9f69-6a4694ecb433-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.855595 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.855569 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hrtcf\" (UniqueName: \"kubernetes.io/projected/a9a4727f-6d8b-4581-9f69-6a4694ecb433-kube-api-access-hrtcf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.855789 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.853977 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9a4727f-6d8b-4581-9f69-6a4694ecb433-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.858520 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.858481 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9a4727f-6d8b-4581-9f69-6a4694ecb433-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.858899 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.858881 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9a4727f-6d8b-4581-9f69-6a4694ecb433-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:00.867817 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:00.867796 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrtcf\" (UniqueName: \"kubernetes.io/projected/a9a4727f-6d8b-4581-9f69-6a4694ecb433-kube-api-access-hrtcf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw\" (UID: \"a9a4727f-6d8b-4581-9f69-6a4694ecb433\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:01.039425 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:01.039320 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:01.180382 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:01.180250 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw"] May 28 03:30:01.183194 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:30:01.183154 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9a4727f_6d8b_4581_9f69_6a4694ecb433.slice/crio-d9c0ac49b6a211f648528e66266077538c90fb75267df06608a0d06bc5e344a2 WatchSource:0}: Error finding container d9c0ac49b6a211f648528e66266077538c90fb75267df06608a0d06bc5e344a2: Status 404 returned error can't find the container with id d9c0ac49b6a211f648528e66266077538c90fb75267df06608a0d06bc5e344a2 May 28 03:30:01.348640 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:01.348536 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" event={"ID":"a9a4727f-6d8b-4581-9f69-6a4694ecb433","Type":"ContainerStarted","Data":"e7b33fe15a83367d8bca8b514af1f154134c1c378bbc724a9ae148b623d12efb"} May 28 03:30:01.348640 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:01.348577 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" event={"ID":"a9a4727f-6d8b-4581-9f69-6a4694ecb433","Type":"ContainerStarted","Data":"d9c0ac49b6a211f648528e66266077538c90fb75267df06608a0d06bc5e344a2"} May 28 03:30:07.373008 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:07.372888 2555 generic.go:358] "Generic (PLEG): container finished" podID="a9a4727f-6d8b-4581-9f69-6a4694ecb433" containerID="e7b33fe15a83367d8bca8b514af1f154134c1c378bbc724a9ae148b623d12efb" exitCode=0 May 28 03:30:07.373008 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:07.372935 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" event={"ID":"a9a4727f-6d8b-4581-9f69-6a4694ecb433","Type":"ContainerDied","Data":"e7b33fe15a83367d8bca8b514af1f154134c1c378bbc724a9ae148b623d12efb"} May 28 03:30:08.377830 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:08.377794 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" event={"ID":"a9a4727f-6d8b-4581-9f69-6a4694ecb433","Type":"ContainerStarted","Data":"04eabfd6eef441a1dbc72c9a5f599f16645d237dee87e343caf65dee81936ea0"} May 28 03:30:08.378298 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:08.378030 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:30:08.397620 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:08.397567 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" podStartSLOduration=8.176292674 podStartE2EDuration="8.397552005s" podCreationTimestamp="2026-05-28 03:30:00 +0000 UTC" firstStartedPulling="2026-05-28 03:30:07.373696695 +0000 UTC m=+800.084342635" lastFinishedPulling="2026-05-28 03:30:07.594956021 +0000 UTC m=+800.305601966" observedRunningTime="2026-05-28 03:30:08.39628724 +0000 UTC m=+801.106933202" watchObservedRunningTime="2026-05-28 03:30:08.397552005 +0000 UTC m=+801.108197966" May 28 03:30:19.394454 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:30:19.394422 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw" May 28 03:31:47.805365 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:31:47.805286 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:31:47.806130 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:31:47.806110 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:32:11.271424 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.271379 2555 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84d8b48b59-vbcqc"] May 28 03:32:11.271891 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.271656 2555 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-84d8b48b59-vbcqc" podUID="be48d37b-56e5-41e2-b72f-503f02226b9e" containerName="manager" containerID="cri-o://a1f9c7714512134872d77294a1151727e97cb7b4fefff00c7207634a8c3682e9" gracePeriod=30 May 28 03:32:11.529176 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.529103 2555 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d8b48b59-vbcqc" May 28 03:32:11.574673 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.574637 2555 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45qv7\" (UniqueName: \"kubernetes.io/projected/be48d37b-56e5-41e2-b72f-503f02226b9e-kube-api-access-45qv7\") pod \"be48d37b-56e5-41e2-b72f-503f02226b9e\" (UID: \"be48d37b-56e5-41e2-b72f-503f02226b9e\") " May 28 03:32:11.576722 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.576683 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be48d37b-56e5-41e2-b72f-503f02226b9e-kube-api-access-45qv7" (OuterVolumeSpecName: "kube-api-access-45qv7") pod "be48d37b-56e5-41e2-b72f-503f02226b9e" (UID: "be48d37b-56e5-41e2-b72f-503f02226b9e"). InnerVolumeSpecName "kube-api-access-45qv7". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 03:32:11.675846 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.675799 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45qv7\" (UniqueName: \"kubernetes.io/projected/be48d37b-56e5-41e2-b72f-503f02226b9e-kube-api-access-45qv7\") on node \"ip-10-0-141-36.ec2.internal\" DevicePath \"\"" May 28 03:32:11.816419 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.816327 2555 generic.go:358] "Generic (PLEG): container finished" podID="be48d37b-56e5-41e2-b72f-503f02226b9e" containerID="a1f9c7714512134872d77294a1151727e97cb7b4fefff00c7207634a8c3682e9" exitCode=0 May 28 03:32:11.816419 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.816396 2555 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d8b48b59-vbcqc" May 28 03:32:11.816634 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.816418 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d8b48b59-vbcqc" event={"ID":"be48d37b-56e5-41e2-b72f-503f02226b9e","Type":"ContainerDied","Data":"a1f9c7714512134872d77294a1151727e97cb7b4fefff00c7207634a8c3682e9"} May 28 03:32:11.816634 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.816468 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d8b48b59-vbcqc" event={"ID":"be48d37b-56e5-41e2-b72f-503f02226b9e","Type":"ContainerDied","Data":"774218a5bacdce18d0d44acd6e858178cd2ba5e54b37a55340d34b16a138f0a2"} May 28 03:32:11.816634 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.816484 2555 scope.go:117] "RemoveContainer" containerID="a1f9c7714512134872d77294a1151727e97cb7b4fefff00c7207634a8c3682e9" May 28 03:32:11.824656 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.824512 2555 scope.go:117] "RemoveContainer" containerID="a1f9c7714512134872d77294a1151727e97cb7b4fefff00c7207634a8c3682e9" May 28 03:32:11.824844 ip-10-0-141-36 kubenswrapper[2555]: E0528 03:32:11.824823 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1f9c7714512134872d77294a1151727e97cb7b4fefff00c7207634a8c3682e9\": container with ID starting with a1f9c7714512134872d77294a1151727e97cb7b4fefff00c7207634a8c3682e9 not found: ID does not exist" containerID="a1f9c7714512134872d77294a1151727e97cb7b4fefff00c7207634a8c3682e9" May 28 03:32:11.824907 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.824858 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1f9c7714512134872d77294a1151727e97cb7b4fefff00c7207634a8c3682e9"} err="failed to get container status \"a1f9c7714512134872d77294a1151727e97cb7b4fefff00c7207634a8c3682e9\": rpc error: code = NotFound desc = could not find container \"a1f9c7714512134872d77294a1151727e97cb7b4fefff00c7207634a8c3682e9\": container with ID starting with a1f9c7714512134872d77294a1151727e97cb7b4fefff00c7207634a8c3682e9 not found: ID does not exist" May 28 03:32:11.840365 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.840332 2555 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84d8b48b59-vbcqc"] May 28 03:32:11.844708 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.844676 2555 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84d8b48b59-vbcqc"] May 28 03:32:11.879357 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:11.879318 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be48d37b-56e5-41e2-b72f-503f02226b9e" path="/var/lib/kubelet/pods/be48d37b-56e5-41e2-b72f-503f02226b9e/volumes" May 28 03:32:12.549072 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:12.549037 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84d8b48b59-2hrbl"] May 28 03:32:12.549536 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:12.549370 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be48d37b-56e5-41e2-b72f-503f02226b9e" containerName="manager" May 28 03:32:12.549536 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:12.549381 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="be48d37b-56e5-41e2-b72f-503f02226b9e" containerName="manager" May 28 03:32:12.549536 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:12.549436 2555 memory_manager.go:356] "RemoveStaleState removing state" podUID="be48d37b-56e5-41e2-b72f-503f02226b9e" containerName="manager" May 28 03:32:12.553626 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:12.553598 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d8b48b59-2hrbl" May 28 03:32:12.556036 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:12.556012 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" May 28 03:32:12.556184 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:12.556012 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m26n2\"" May 28 03:32:12.561057 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:12.561031 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d8b48b59-2hrbl"] May 28 03:32:12.584899 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:12.584864 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7xsg\" (UniqueName: \"kubernetes.io/projected/f68438b5-9194-4ef4-bef7-7305a12bf252-kube-api-access-q7xsg\") pod \"maas-controller-84d8b48b59-2hrbl\" (UID: \"f68438b5-9194-4ef4-bef7-7305a12bf252\") " pod="opendatahub/maas-controller-84d8b48b59-2hrbl" May 28 03:32:12.686384 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:12.686339 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q7xsg\" (UniqueName: \"kubernetes.io/projected/f68438b5-9194-4ef4-bef7-7305a12bf252-kube-api-access-q7xsg\") pod \"maas-controller-84d8b48b59-2hrbl\" (UID: \"f68438b5-9194-4ef4-bef7-7305a12bf252\") " pod="opendatahub/maas-controller-84d8b48b59-2hrbl" May 28 03:32:12.695017 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:12.694986 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7xsg\" (UniqueName: \"kubernetes.io/projected/f68438b5-9194-4ef4-bef7-7305a12bf252-kube-api-access-q7xsg\") pod \"maas-controller-84d8b48b59-2hrbl\" (UID: \"f68438b5-9194-4ef4-bef7-7305a12bf252\") " pod="opendatahub/maas-controller-84d8b48b59-2hrbl" May 28 03:32:12.864337 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:12.864245 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d8b48b59-2hrbl" May 28 03:32:12.998539 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:12.998466 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d8b48b59-2hrbl"] May 28 03:32:13.001441 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:32:13.001410 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf68438b5_9194_4ef4_bef7_7305a12bf252.slice/crio-a5e610cd0188b30938ef1ecb67fc18364dfb4ea56882c89166e58cad699580d4 WatchSource:0}: Error finding container a5e610cd0188b30938ef1ecb67fc18364dfb4ea56882c89166e58cad699580d4: Status 404 returned error can't find the container with id a5e610cd0188b30938ef1ecb67fc18364dfb4ea56882c89166e58cad699580d4 May 28 03:32:13.825330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:13.825241 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d8b48b59-2hrbl" event={"ID":"f68438b5-9194-4ef4-bef7-7305a12bf252","Type":"ContainerStarted","Data":"e5140536b43069e551cb9e841e21ff9ab6aa248b3a6e898d5f02a0551fcdb2b0"} May 28 03:32:13.825330 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:13.825288 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d8b48b59-2hrbl" event={"ID":"f68438b5-9194-4ef4-bef7-7305a12bf252","Type":"ContainerStarted","Data":"a5e610cd0188b30938ef1ecb67fc18364dfb4ea56882c89166e58cad699580d4"} May 28 03:32:13.825750 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:13.825382 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84d8b48b59-2hrbl" May 28 03:32:13.863093 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:13.863037 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-84d8b48b59-2hrbl" podStartSLOduration=1.36356894 podStartE2EDuration="1.863019771s" podCreationTimestamp="2026-05-28 03:32:12 +0000 UTC" firstStartedPulling="2026-05-28 03:32:13.003127673 +0000 UTC m=+925.713773613" lastFinishedPulling="2026-05-28 03:32:13.502578504 +0000 UTC m=+926.213224444" observedRunningTime="2026-05-28 03:32:13.861474123 +0000 UTC m=+926.572120084" watchObservedRunningTime="2026-05-28 03:32:13.863019771 +0000 UTC m=+926.573665732" May 28 03:32:24.834146 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:32:24.834113 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84d8b48b59-2hrbl" May 28 03:36:47.829407 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:36:47.829374 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:36:47.831099 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:36:47.831080 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:41:47.854334 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:41:47.854302 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:41:47.855977 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:41:47.855953 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:46:47.876886 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:46:47.876854 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:46:47.881170 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:46:47.881148 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:51:47.901275 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:51:47.901239 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:51:47.907706 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:51:47.907680 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:53:26.243374 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:26.243335 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-4rpvb_568e1f50-02ca-4f32-b096-78b44cbc6b4d/manager/0.log" May 28 03:53:26.359025 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:26.358994 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6f94d9d45c-bmpnb_b6c40d20-0624-4dae-96d0-e4ef7169a39b/maas-api/0.log" May 28 03:53:26.692339 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:26.692308 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84d8b48b59-2hrbl_f68438b5-9194-4ef4-bef7-7305a12bf252/manager/0.log" May 28 03:53:26.807429 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:26.807393 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-hvzw9_0ed3886f-9b44-40e8-a0ea-32171686ff9f/manager/2.log" May 28 03:53:26.925389 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:26.925357 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77cd789dcd-2fnxh_ac7ef498-9c08-40af-a053-34db052d2f91/manager/0.log" May 28 03:53:28.593438 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:28.593387 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-tnn8l_9733288e-7ee5-4015-b1a0-ec6c948dc7b4/manager/0.log" May 28 03:53:28.915954 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:28.915918 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-rlfk4_993a9ccc-e9ed-402b-b76f-92a749da0996/registry-server/0.log" May 28 03:53:29.265138 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:29.265054 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-bvn4k_2a3c1b6d-a16c-4c2a-85e3-0d07912e5892/manager/0.log" May 28 03:53:29.708186 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:29.708158 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-9hdvs_693c2353-4c65-4fac-bc7a-89aca2ae2f41/discovery/0.log" May 28 03:53:29.811241 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:29.811183 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b78c488db-7qn7h_34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92/kube-auth-proxy/0.log" May 28 03:53:30.133914 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:30.133839 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-hvdwr_65d3c4ce-2c21-4961-9472-a822c5e43d52/payload-processing/0.log" May 28 03:53:30.894876 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:30.894841 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw_a9a4727f-6d8b-4581-9f69-6a4694ecb433/storage-initializer/0.log" May 28 03:53:30.902003 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:30.901973 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccl6slw_a9a4727f-6d8b-4581-9f69-6a4694ecb433/main/0.log" May 28 03:53:31.119203 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:31.119172 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-94v5c_ff537ae8-ac1e-4c0c-87d9-47805ccfe77b/storage-initializer/0.log" May 28 03:53:31.126870 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:31.126828 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-94v5c_ff537ae8-ac1e-4c0c-87d9-47805ccfe77b/main/0.log" May 28 03:53:37.881551 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:37.881518 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pkkqh_06e44704-03d7-4840-a298-191be4e1b255/global-pull-secret-syncer/0.log" May 28 03:53:37.989601 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:37.989566 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lp9sw_ce5ad371-6b1d-464c-8aa8-6b1b0339de53/konnectivity-agent/0.log" May 28 03:53:38.138725 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:38.138641 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-36.ec2.internal_5f5996663d36d1c198c700af5357120a/haproxy/0.log" May 28 03:53:42.546446 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:42.546410 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-tnn8l_9733288e-7ee5-4015-b1a0-ec6c948dc7b4/manager/0.log" May 28 03:53:42.648002 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:42.647966 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-rlfk4_993a9ccc-e9ed-402b-b76f-92a749da0996/registry-server/0.log" May 28 03:53:42.859129 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:42.859043 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-bvn4k_2a3c1b6d-a16c-4c2a-85e3-0d07912e5892/manager/0.log" May 28 03:53:44.815659 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:44.815625 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rh2gp_1dce139a-e95b-498c-abdd-81dcc70e0d2b/node-exporter/0.log" May 28 03:53:44.835754 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:44.835728 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rh2gp_1dce139a-e95b-498c-abdd-81dcc70e0d2b/kube-rbac-proxy/0.log" May 28 03:53:44.858222 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:44.858195 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rh2gp_1dce139a-e95b-498c-abdd-81dcc70e0d2b/init-textfile/0.log" May 28 03:53:46.424907 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.424871 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-pv6vg_7497ac51-1873-4881-9f5d-fd40ca01efd9/networking-console-plugin/0.log" May 28 03:53:46.622394 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.622357 2555 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn"] May 28 03:53:46.625030 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.625006 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.627517 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.627482 2555 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-q6qd7\"/\"default-dockercfg-lbjv5\"" May 28 03:53:46.627643 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.627552 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q6qd7\"/\"kube-root-ca.crt\"" May 28 03:53:46.628402 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.628383 2555 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q6qd7\"/\"openshift-service-ca.crt\"" May 28 03:53:46.636486 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.636460 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn"] May 28 03:53:46.724839 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.724738 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8039a0e7-b62f-41ec-8e47-8c26be24e373-sys\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.724839 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.724814 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8039a0e7-b62f-41ec-8e47-8c26be24e373-proc\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.724839 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.724840 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf5jr\" (UniqueName: \"kubernetes.io/projected/8039a0e7-b62f-41ec-8e47-8c26be24e373-kube-api-access-rf5jr\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.725078 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.724865 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8039a0e7-b62f-41ec-8e47-8c26be24e373-lib-modules\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.725078 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.724892 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8039a0e7-b62f-41ec-8e47-8c26be24e373-podres\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.825712 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.825676 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8039a0e7-b62f-41ec-8e47-8c26be24e373-proc\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.825712 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.825714 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rf5jr\" (UniqueName: \"kubernetes.io/projected/8039a0e7-b62f-41ec-8e47-8c26be24e373-kube-api-access-rf5jr\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.825929 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.825740 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8039a0e7-b62f-41ec-8e47-8c26be24e373-lib-modules\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.825929 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.825804 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8039a0e7-b62f-41ec-8e47-8c26be24e373-proc\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.825929 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.825846 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8039a0e7-b62f-41ec-8e47-8c26be24e373-lib-modules\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.825929 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.825854 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8039a0e7-b62f-41ec-8e47-8c26be24e373-podres\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.825929 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.825909 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8039a0e7-b62f-41ec-8e47-8c26be24e373-podres\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.825929 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.825910 2555 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8039a0e7-b62f-41ec-8e47-8c26be24e373-sys\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.826112 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.825986 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8039a0e7-b62f-41ec-8e47-8c26be24e373-sys\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.834346 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.834310 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf5jr\" (UniqueName: \"kubernetes.io/projected/8039a0e7-b62f-41ec-8e47-8c26be24e373-kube-api-access-rf5jr\") pod \"perf-node-gather-daemonset-b9zhn\" (UID: \"8039a0e7-b62f-41ec-8e47-8c26be24e373\") " pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:46.935486 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:46.935442 2555 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:47.079756 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:47.079732 2555 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn"] May 28 03:53:47.082737 ip-10-0-141-36 kubenswrapper[2555]: W0528 03:53:47.082704 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8039a0e7_b62f_41ec_8e47_8c26be24e373.slice/crio-d21c57d6aa4cd573daf3581876251c3a057d5f10d86d424f296e30b9836a7379 WatchSource:0}: Error finding container d21c57d6aa4cd573daf3581876251c3a057d5f10d86d424f296e30b9836a7379: Status 404 returned error can't find the container with id d21c57d6aa4cd573daf3581876251c3a057d5f10d86d424f296e30b9836a7379 May 28 03:53:47.084361 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:47.084343 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 03:53:47.429181 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:47.429134 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" event={"ID":"8039a0e7-b62f-41ec-8e47-8c26be24e373","Type":"ContainerStarted","Data":"e8ae47fc3e55be3841b00efd8f60e3f92c1c2b4899c6871bb23c82936b559a87"} May 28 03:53:47.429181 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:47.429177 2555 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" event={"ID":"8039a0e7-b62f-41ec-8e47-8c26be24e373","Type":"ContainerStarted","Data":"d21c57d6aa4cd573daf3581876251c3a057d5f10d86d424f296e30b9836a7379"} May 28 03:53:47.429695 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:47.429249 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:47.444867 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:47.444810 2555 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" podStartSLOduration=1.444790023 podStartE2EDuration="1.444790023s" podCreationTimestamp="2026-05-28 03:53:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 03:53:47.444372641 +0000 UTC m=+2220.155018601" watchObservedRunningTime="2026-05-28 03:53:47.444790023 +0000 UTC m=+2220.155435986" May 28 03:53:48.933442 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:48.933413 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kcl9f_ec456511-caf8-4385-909b-7c7ba33e1b1f/dns/0.log" May 28 03:53:48.957035 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:48.957004 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kcl9f_ec456511-caf8-4385-909b-7c7ba33e1b1f/kube-rbac-proxy/0.log" May 28 03:53:49.083232 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:49.083204 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gbftm_9d2867de-afec-49ff-be90-2680f09ddc2e/dns-node-resolver/0.log" May 28 03:53:49.615122 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:49.615085 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zx6tr_900e522d-e9a0-4344-ad62-c0518eb0cba7/node-ca/0.log" May 28 03:53:50.588430 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:50.588397 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-9hdvs_693c2353-4c65-4fac-bc7a-89aca2ae2f41/discovery/0.log" May 28 03:53:50.618565 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:50.618524 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b78c488db-7qn7h_34c4d4c6-6a98-42c0-b0f1-8e4a0ac77f92/kube-auth-proxy/0.log" May 28 03:53:50.811829 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:50.811794 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-hvdwr_65d3c4ce-2c21-4961-9472-a822c5e43d52/payload-processing/0.log" May 28 03:53:51.382656 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:51.382631 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8sbsk_e6747510-251a-41cb-adef-fc02441f4e00/serve-healthcheck-canary/0.log" May 28 03:53:52.074049 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:52.074019 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-849pb_9f74532d-82aa-461a-877f-9be2ca692e9e/kube-rbac-proxy/0.log" May 28 03:53:52.094129 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:52.094100 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-849pb_9f74532d-82aa-461a-877f-9be2ca692e9e/exporter/0.log" May 28 03:53:52.115795 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:52.115765 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-849pb_9f74532d-82aa-461a-877f-9be2ca692e9e/extractor/0.log" May 28 03:53:53.442600 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:53.442573 2555 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-q6qd7/perf-node-gather-daemonset-b9zhn" May 28 03:53:54.161953 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:54.161925 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-4rpvb_568e1f50-02ca-4f32-b096-78b44cbc6b4d/manager/0.log" May 28 03:53:54.203365 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:54.203320 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6f94d9d45c-bmpnb_b6c40d20-0624-4dae-96d0-e4ef7169a39b/maas-api/0.log" May 28 03:53:54.360013 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:54.359969 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84d8b48b59-2hrbl_f68438b5-9194-4ef4-bef7-7305a12bf252/manager/0.log" May 28 03:53:54.380839 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:54.380808 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-hvzw9_0ed3886f-9b44-40e8-a0ea-32171686ff9f/manager/1.log" May 28 03:53:54.401770 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:54.401739 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-hvzw9_0ed3886f-9b44-40e8-a0ea-32171686ff9f/manager/2.log" May 28 03:53:54.451042 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:53:54.450965 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77cd789dcd-2fnxh_ac7ef498-9c08-40af-a053-34db052d2f91/manager/0.log" May 28 03:54:01.513071 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:01.513042 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7zh5q_f7a4380c-a918-4aed-ab01-ab654612890d/kube-multus/0.log" May 28 03:54:01.533686 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:01.533658 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4hk25_d3e7c7f4-9f55-4d7e-9f01-7025793968a6/kube-multus-additional-cni-plugins/0.log" May 28 03:54:01.553637 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:01.553598 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4hk25_d3e7c7f4-9f55-4d7e-9f01-7025793968a6/egress-router-binary-copy/0.log" May 28 03:54:01.572944 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:01.572913 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4hk25_d3e7c7f4-9f55-4d7e-9f01-7025793968a6/cni-plugins/0.log" May 28 03:54:01.594780 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:01.594752 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4hk25_d3e7c7f4-9f55-4d7e-9f01-7025793968a6/bond-cni-plugin/0.log" May 28 03:54:01.613882 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:01.613858 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4hk25_d3e7c7f4-9f55-4d7e-9f01-7025793968a6/routeoverride-cni/0.log" May 28 03:54:01.633318 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:01.633294 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4hk25_d3e7c7f4-9f55-4d7e-9f01-7025793968a6/whereabouts-cni-bincopy/0.log" May 28 03:54:01.673879 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:01.673852 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4hk25_d3e7c7f4-9f55-4d7e-9f01-7025793968a6/whereabouts-cni/0.log" May 28 03:54:02.109613 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:02.109577 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mkdkv_f8725dbf-86ef-41e7-845a-5b81d2343b2b/network-metrics-daemon/0.log" May 28 03:54:02.131719 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:02.131690 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mkdkv_f8725dbf-86ef-41e7-845a-5b81d2343b2b/kube-rbac-proxy/0.log" May 28 03:54:03.521612 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:03.521576 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-controller/0.log" May 28 03:54:03.536282 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:03.536243 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/0.log" May 28 03:54:03.555958 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:03.555924 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovn-acl-logging/1.log" May 28 03:54:03.577128 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:03.577105 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/kube-rbac-proxy-node/0.log" May 28 03:54:03.598420 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:03.598388 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/kube-rbac-proxy-ovn-metrics/0.log" May 28 03:54:03.613691 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:03.613662 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/northd/0.log" May 28 03:54:03.632350 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:03.632331 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/nbdb/0.log" May 28 03:54:03.652174 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:03.652149 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/sbdb/0.log" May 28 03:54:03.825864 ip-10-0-141-36 kubenswrapper[2555]: I0528 03:54:03.825775 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqxrv_ec36f7a6-5a79-46b3-b2c4-c9ff7ecea720/ovnkube-controller/0.log"