May 20 18:59:02.308614 ip-10-0-138-66 systemd[1]: Starting Kubernetes Kubelet... May 20 18:59:02.825779 ip-10-0-138-66 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 20 18:59:02.825779 ip-10-0-138-66 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 20 18:59:02.825779 ip-10-0-138-66 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 20 18:59:02.825779 ip-10-0-138-66 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 20 18:59:02.825779 ip-10-0-138-66 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 20 18:59:02.828278 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.828180 2575 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 20 18:59:02.832239 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832224 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 20 18:59:02.832239 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832238 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832242 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832245 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832248 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832252 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832255 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832258 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832262 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832265 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832268 2575 feature_gate.go:328] unrecognized feature gate: DualReplica May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832271 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832274 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832277 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832279 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832282 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832285 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832287 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832290 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832292 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832295 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 20 18:59:02.832300 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832297 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832302 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832306 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832309 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832312 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832315 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832318 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832321 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832324 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832327 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832329 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832338 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832341 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832343 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832347 2575 feature_gate.go:328] unrecognized feature gate: NewOLM May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832350 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832352 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832355 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832357 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832359 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 20 18:59:02.832771 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832362 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832364 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832367 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832369 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832372 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832374 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832377 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832379 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832382 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832384 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832387 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832389 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832392 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832394 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832399 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832403 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832406 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832409 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832412 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 20 18:59:02.833533 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832419 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832422 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832424 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832427 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832429 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832438 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832440 2575 feature_gate.go:328] unrecognized feature gate: Example2 May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832443 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832446 2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832448 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832451 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832453 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832456 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832458 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832461 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832464 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832466 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832469 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832471 2575 feature_gate.go:328] unrecognized feature gate: Example May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832473 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 20 18:59:02.834146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832476 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832478 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832481 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832484 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832486 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.832488 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833326 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833344 2575 feature_gate.go:328] unrecognized feature gate: DualReplica May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833350 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833354 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833360 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833364 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833368 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833373 2575 feature_gate.go:328] unrecognized feature gate: Example May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833383 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833388 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833393 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833399 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833403 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833407 2575 feature_gate.go:328] unrecognized feature gate: NewOLM May 20 18:59:02.834661 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833413 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833418 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833422 2575 feature_gate.go:328] unrecognized feature gate: Example2 May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833426 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833431 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833435 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833440 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833448 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833453 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833458 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833463 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833468 2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833472 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833477 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833481 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833486 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833490 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833495 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833500 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 20 18:59:02.835181 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833507 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833516 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833522 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833526 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833531 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833535 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833539 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833543 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833547 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833552 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833556 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833561 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833566 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833575 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833580 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833587 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833591 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833595 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833600 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833605 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 20 18:59:02.835699 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833609 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833614 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833618 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833622 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833627 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833636 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833643 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833649 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833655 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833660 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833665 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833669 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833674 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833679 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833684 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833689 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833694 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833703 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833708 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 20 18:59:02.836175 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833712 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833721 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833728 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833746 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833751 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833756 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833760 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833774 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833796 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833802 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833807 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833813 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833818 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.833823 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834460 2575 flags.go:64] FLAG: --address="0.0.0.0" May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834484 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834492 2575 flags.go:64] FLAG: --anonymous-auth="true" May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834497 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834501 2575 flags.go:64] FLAG: --authentication-token-webhook="false" May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834505 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 20 18:59:02.836653 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834509 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834514 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834518 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834521 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834524 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834528 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834531 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834534 2575 flags.go:64] FLAG: --cgroup-root="" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834537 2575 flags.go:64] FLAG: --cgroups-per-qos="true" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834540 2575 flags.go:64] FLAG: --client-ca-file="" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834543 2575 flags.go:64] FLAG: --cloud-config="" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834545 2575 flags.go:64] FLAG: --cloud-provider="external" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834548 2575 flags.go:64] FLAG: --cluster-dns="[]" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834553 2575 flags.go:64] FLAG: --cluster-domain="" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834556 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834559 2575 flags.go:64] FLAG: --config-dir="" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834562 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834566 2575 flags.go:64] FLAG: --container-log-max-files="5" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834570 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834574 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834577 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834580 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834583 2575 flags.go:64] FLAG: --contention-profiling="false" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834586 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" May 20 18:59:02.837147 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834589 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834592 2575 flags.go:64] FLAG: --cpu-manager-policy="none" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834595 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834600 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834604 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834607 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834610 2575 flags.go:64] FLAG: --enable-load-reader="false" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834613 2575 flags.go:64] FLAG: --enable-server="true" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834616 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834621 2575 flags.go:64] FLAG: --event-burst="100" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834624 2575 flags.go:64] FLAG: --event-qps="50" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834627 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834630 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834634 2575 flags.go:64] FLAG: --eviction-hard="" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834637 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834640 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834644 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834646 2575 flags.go:64] FLAG: --eviction-soft="" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834649 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834652 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834655 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834658 2575 flags.go:64] FLAG: --experimental-mounter-path="" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834661 2575 flags.go:64] FLAG: --fail-cgroupv1="false" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834664 2575 flags.go:64] FLAG: --fail-swap-on="true" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834667 2575 flags.go:64] FLAG: --feature-gates="" May 20 18:59:02.837752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834671 2575 flags.go:64] FLAG: --file-check-frequency="20s" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834675 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834678 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834681 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834684 2575 flags.go:64] FLAG: --healthz-port="10248" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834687 2575 flags.go:64] FLAG: --help="false" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834690 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-138-66.ec2.internal" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834693 2575 flags.go:64] FLAG: --housekeeping-interval="10s" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834696 2575 flags.go:64] FLAG: --http-check-frequency="20s" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834699 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834704 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834708 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834711 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834714 2575 flags.go:64] FLAG: --image-service-endpoint="" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834717 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834720 2575 flags.go:64] FLAG: --kube-api-burst="100" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834723 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834727 2575 flags.go:64] FLAG: --kube-api-qps="50" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834730 2575 flags.go:64] FLAG: --kube-reserved="" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834733 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834735 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834738 2575 flags.go:64] FLAG: --kubelet-cgroups="" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834741 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834744 2575 flags.go:64] FLAG: --lock-file="" May 20 18:59:02.838409 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834747 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834750 2575 flags.go:64] FLAG: --log-flush-frequency="5s" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834753 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834758 2575 flags.go:64] FLAG: --log-json-split-stream="false" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834761 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834764 2575 flags.go:64] FLAG: --log-text-split-stream="false" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834767 2575 flags.go:64] FLAG: --logging-format="text" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834770 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834773 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834776 2575 flags.go:64] FLAG: --manifest-url="" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834779 2575 flags.go:64] FLAG: --manifest-url-header="" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834783 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834786 2575 flags.go:64] FLAG: --max-open-files="1000000" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834790 2575 flags.go:64] FLAG: --max-pods="110" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834793 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834796 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834799 2575 flags.go:64] FLAG: --memory-manager-policy="None" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834802 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834807 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834810 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834814 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834821 2575 flags.go:64] FLAG: --node-status-max-images="50" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834824 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834827 2575 flags.go:64] FLAG: --oom-score-adj="-999" May 20 18:59:02.838983 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834830 2575 flags.go:64] FLAG: --pod-cidr="" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834834 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834839 2575 flags.go:64] FLAG: --pod-manifest-path="" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834842 2575 flags.go:64] FLAG: --pod-max-pids="-1" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834845 2575 flags.go:64] FLAG: --pods-per-core="0" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834848 2575 flags.go:64] FLAG: --port="10250" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834851 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834854 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e523e0cbce6fee61" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834858 2575 flags.go:64] FLAG: --qos-reserved="" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834861 2575 flags.go:64] FLAG: --read-only-port="10255" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834863 2575 flags.go:64] FLAG: --register-node="true" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834867 2575 flags.go:64] FLAG: --register-schedulable="true" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834869 2575 flags.go:64] FLAG: --register-with-taints="" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834873 2575 flags.go:64] FLAG: --registry-burst="10" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834876 2575 flags.go:64] FLAG: --registry-qps="5" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834879 2575 flags.go:64] FLAG: --reserved-cpus="" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834881 2575 flags.go:64] FLAG: --reserved-memory="" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834885 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834888 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834891 2575 flags.go:64] FLAG: --rotate-certificates="false" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834894 2575 flags.go:64] FLAG: --rotate-server-certificates="false" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834897 2575 flags.go:64] FLAG: --runonce="false" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834900 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834903 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834906 2575 flags.go:64] FLAG: --seccomp-default="false" May 20 18:59:02.839570 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834909 2575 flags.go:64] FLAG: --serialize-image-pulls="true" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834912 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834916 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834919 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834922 2575 flags.go:64] FLAG: --storage-driver-password="root" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834925 2575 flags.go:64] FLAG: --storage-driver-secure="false" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834927 2575 flags.go:64] FLAG: --storage-driver-table="stats" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834930 2575 flags.go:64] FLAG: --storage-driver-user="root" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834933 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834936 2575 flags.go:64] FLAG: --sync-frequency="1m0s" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834939 2575 flags.go:64] FLAG: --system-cgroups="" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834960 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834968 2575 flags.go:64] FLAG: --system-reserved-cgroup="" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834971 2575 flags.go:64] FLAG: --tls-cert-file="" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834975 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834978 2575 flags.go:64] FLAG: --tls-min-version="" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834981 2575 flags.go:64] FLAG: --tls-private-key-file="" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834984 2575 flags.go:64] FLAG: --topology-manager-policy="none" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834986 2575 flags.go:64] FLAG: --topology-manager-policy-options="" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834989 2575 flags.go:64] FLAG: --topology-manager-scope="container" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834992 2575 flags.go:64] FLAG: --v="2" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.834997 2575 flags.go:64] FLAG: --version="false" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.835001 2575 flags.go:64] FLAG: --vmodule="" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.835005 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.835008 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 20 18:59:02.840146 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835104 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835107 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835111 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835113 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835120 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835123 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835126 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835129 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835131 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835149 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835154 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835158 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835161 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835164 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835167 2575 feature_gate.go:328] unrecognized feature gate: Example May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835170 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835173 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835175 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835178 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835181 2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 20 18:59:02.840766 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835184 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835186 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835189 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835191 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835194 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835197 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835199 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835215 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835217 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835220 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835223 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835226 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835228 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835231 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835234 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835236 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835239 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835242 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835245 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 20 18:59:02.841299 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835247 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835250 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835253 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835256 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835258 2575 feature_gate.go:328] unrecognized feature gate: Example2 May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835261 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835263 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835266 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835269 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835271 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835274 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835283 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835287 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835290 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835294 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835298 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835301 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835303 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835306 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 20 18:59:02.842102 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835309 2575 feature_gate.go:328] unrecognized feature gate: DualReplica May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835311 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835314 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835316 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835319 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835323 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835326 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835329 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835332 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835335 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835337 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835340 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835343 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835346 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835349 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835352 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835355 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835358 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835360 2575 feature_gate.go:328] unrecognized feature gate: NewOLM May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835363 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 20 18:59:02.842985 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835365 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835368 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835370 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835373 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835375 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835378 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835381 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.835383 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.836033 2575 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.843291 2575 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.843311 2575 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843381 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843389 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843394 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843400 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843404 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 20 18:59:02.843859 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843409 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843413 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843419 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843423 2575 feature_gate.go:328] unrecognized feature gate: Example2 May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843427 2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843432 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843436 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843440 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843445 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843449 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843453 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843458 2575 feature_gate.go:328] unrecognized feature gate: Example May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843462 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843466 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843471 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843476 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843480 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843484 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843488 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843493 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 20 18:59:02.844638 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843497 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843501 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843508 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843515 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843520 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843524 2575 feature_gate.go:328] unrecognized feature gate: DualReplica May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843531 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843536 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843541 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843545 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843549 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843553 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843557 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843561 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843568 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843574 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843579 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843584 2575 feature_gate.go:328] unrecognized feature gate: NewOLM May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843588 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843592 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 20 18:59:02.845133 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843597 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843602 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843606 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843610 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843614 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843618 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843622 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843626 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843631 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843635 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843640 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843645 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843649 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843653 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843657 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843661 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843665 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843670 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843675 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 20 18:59:02.845714 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843685 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843690 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843694 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843698 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843702 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843706 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843710 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843715 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843719 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843723 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843728 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843732 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843736 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843740 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843744 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843749 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843753 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843757 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843761 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 20 18:59:02.846347 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843765 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843769 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843774 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.843782 2575 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843941 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843948 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843953 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843958 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843963 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843967 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843972 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843976 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843981 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843985 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843991 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 20 18:59:02.846913 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.843996 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844000 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844005 2575 feature_gate.go:328] unrecognized feature gate: Example2 May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844008 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844012 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844017 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844021 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844025 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844029 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844033 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844037 2575 feature_gate.go:328] unrecognized feature gate: Example May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844041 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844046 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844050 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844053 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844057 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844061 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844065 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844069 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844073 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 20 18:59:02.847310 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844077 2575 feature_gate.go:328] unrecognized feature gate: NewOLM May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844082 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844086 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844090 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844094 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844098 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844102 2575 feature_gate.go:328] unrecognized feature gate: DualReplica May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844106 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844110 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844114 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844119 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844123 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844128 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844133 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844137 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844143 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844149 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844153 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844157 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844162 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 20 18:59:02.847847 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844166 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844170 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844174 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844178 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844182 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844186 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844190 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844195 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844199 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844224 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844228 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844232 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844237 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844240 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844244 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844248 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844252 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844258 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844263 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 20 18:59:02.848473 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844268 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844272 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844276 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844280 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844284 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844288 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844292 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844296 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844300 2575 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844304 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844308 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844311 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844315 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844320 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844325 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 20 18:59:02.849290 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:02.844328 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 20 18:59:02.849713 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.844336 2575 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 20 18:59:02.849713 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.845264 2575 server.go:962] "Client rotation is on, will bootstrap in background" May 20 18:59:02.850099 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.850083 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 20 18:59:02.851246 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.851232 2575 server.go:1019] "Starting client certificate rotation" May 20 18:59:02.851350 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.851334 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 20 18:59:02.851384 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.851374 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 20 18:59:02.879729 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.879710 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 20 18:59:02.882321 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.882302 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 20 18:59:02.898616 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.898589 2575 log.go:25] "Validated CRI v1 runtime API" May 20 18:59:02.905192 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.905170 2575 log.go:25] "Validated CRI v1 image API" May 20 18:59:02.906497 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.906474 2575 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 20 18:59:02.909812 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.909791 2575 fs.go:135] Filesystem UUIDs: map[4ec226ac-0152-4c38-988d-8ccd3747b3c0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a59480dc-8e49-4c34-945e-1e11aa729cfe:/dev/nvme0n1p3] May 20 18:59:02.909882 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.909812 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 20 18:59:02.912023 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.912000 2575 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 20 18:59:02.915420 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.915309 2575 manager.go:217] Machine: {Timestamp:2026-05-20 18:59:02.914797911 +0000 UTC m=+0.460756758 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3110063 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d7155d1aead7f9256ab1e8ae62767 SystemUUID:ec2d7155-d1ae-ad7f-9256-ab1e8ae62767 BootID:a512e1ea-7252-4b4e-8003-acea6b2cc912 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3c:20:7d:e1:db Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3c:20:7d:e1:db Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:27:3e:88:35:20 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 20 18:59:02.916279 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.916267 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 20 18:59:02.916418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.916404 2575 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 20 18:59:02.919247 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.919217 2575 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 20 18:59:02.919445 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.919250 2575 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-66.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 20 18:59:02.919492 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.919456 2575 topology_manager.go:138] "Creating topology manager with none policy" May 20 18:59:02.919492 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.919464 2575 container_manager_linux.go:306] "Creating device plugin manager" May 20 18:59:02.919492 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.919472 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 20 18:59:02.920360 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.920348 2575 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 20 18:59:02.922680 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.922667 2575 state_mem.go:36] "Initialized new in-memory state store" May 20 18:59:02.922811 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.922802 2575 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 20 18:59:02.925596 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.925584 2575 kubelet.go:491] "Attempting to sync node with API server" May 20 18:59:02.925641 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.925599 2575 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 20 18:59:02.925641 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.925612 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 20 18:59:02.925641 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.925621 2575 kubelet.go:397] "Adding apiserver pod source" May 20 18:59:02.925641 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.925631 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 20 18:59:02.926822 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.926808 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 20 18:59:02.926905 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.926833 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 20 18:59:02.930533 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.930515 2575 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 20 18:59:02.932776 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.932760 2575 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 20 18:59:02.933693 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.933682 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 20 18:59:02.933734 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.933698 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 20 18:59:02.933734 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.933705 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 20 18:59:02.933734 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.933710 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 20 18:59:02.933734 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.933719 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 20 18:59:02.933734 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.933724 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 20 18:59:02.933734 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.933730 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 20 18:59:02.933734 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.933736 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 20 18:59:02.933924 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.933742 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 20 18:59:02.933924 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.933748 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 20 18:59:02.933924 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.933762 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 20 18:59:02.934233 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.934225 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 20 18:59:02.936599 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.936586 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 20 18:59:02.936599 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.936598 2575 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 20 18:59:02.937759 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:02.937725 2575 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-66.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 20 18:59:02.937853 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:02.937782 2575 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 20 18:59:02.938047 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.938030 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-66nsh" May 20 18:59:02.939966 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.939838 2575 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 20 18:59:02.940033 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.939990 2575 server.go:1295] "Started kubelet" May 20 18:59:02.940562 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.940502 2575 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 20 18:59:02.940668 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.940585 2575 server_v1.go:47] "podresources" method="list" useActivePods=true May 20 18:59:02.940668 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.940572 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 20 18:59:02.940901 ip-10-0-138-66 systemd[1]: Started Kubernetes Kubelet. May 20 18:59:02.944384 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.944368 2575 server.go:317] "Adding debug handlers to kubelet server" May 20 18:59:02.944630 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.944607 2575 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 20 18:59:02.949038 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.949017 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-66nsh" May 20 18:59:02.951791 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.951771 2575 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-66.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 20 18:59:02.951884 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:02.951864 2575 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 20 18:59:02.952821 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:02.951658 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-66.ec2.internal.18b15b252887dc64 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-66.ec2.internal,UID:ip-10-0-138-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-66.ec2.internal,},FirstTimestamp:2026-05-20 18:59:02.939966564 +0000 UTC m=+0.485925409,LastTimestamp:2026-05-20 18:59:02.939966564 +0000 UTC m=+0.485925409,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-66.ec2.internal,}" May 20 18:59:02.954447 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.954432 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 20 18:59:02.954519 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.954462 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 20 18:59:02.954576 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.954540 2575 volume_manager.go:295] "The desired_state_of_world populator starts" May 20 18:59:02.954576 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.954552 2575 volume_manager.go:297] "Starting Kubelet Volume Manager" May 20 18:59:02.954650 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.954598 2575 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 20 18:59:02.954695 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.954678 2575 reconstruct.go:97] "Volume reconstruction finished" May 20 18:59:02.954695 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.954687 2575 reconciler.go:26] "Reconciler: start to sync state" May 20 18:59:02.955352 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.955333 2575 factory.go:55] Registering systemd factory May 20 18:59:02.955440 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.955396 2575 factory.go:223] Registration of the systemd container factory successfully May 20 18:59:02.955606 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:02.955436 2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-66.ec2.internal\" not found" May 20 18:59:02.955718 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.955618 2575 factory.go:153] Registering CRI-O factory May 20 18:59:02.955718 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.955634 2575 factory.go:223] Registration of the crio container factory successfully May 20 18:59:02.955718 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.955684 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 20 18:59:02.955718 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.955704 2575 factory.go:103] Registering Raw factory May 20 18:59:02.955718 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.955718 2575 manager.go:1196] Started watching for new ooms in manager May 20 18:59:02.956037 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.956023 2575 manager.go:319] Starting recovery of all containers May 20 18:59:02.960239 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.960193 2575 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 20 18:59:02.962852 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:02.962830 2575 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-66.ec2.internal\" not found" node="ip-10-0-138-66.ec2.internal" May 20 18:59:02.966866 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.966851 2575 manager.go:324] Recovery completed May 20 18:59:02.971131 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.971118 2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 20 18:59:02.973037 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.973021 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasSufficientMemory" May 20 18:59:02.973114 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.973056 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasNoDiskPressure" May 20 18:59:02.973114 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.973072 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasSufficientPID" May 20 18:59:02.973630 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.973616 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" May 20 18:59:02.973698 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.973630 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 20 18:59:02.973698 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.973650 2575 state_mem.go:36] "Initialized new in-memory state store" May 20 18:59:02.977650 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.977637 2575 policy_none.go:49] "None policy: Start" May 20 18:59:02.977685 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.977654 2575 memory_manager.go:186] "Starting memorymanager" policy="None" May 20 18:59:02.977685 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:02.977664 2575 state_mem.go:35] "Initializing new in-memory state store" May 20 18:59:03.016791 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.016773 2575 manager.go:341] "Starting Device Plugin manager" May 20 18:59:03.041013 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.016861 2575 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 20 18:59:03.041013 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.016875 2575 server.go:85] "Starting device plugin registration server" May 20 18:59:03.041013 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.017089 2575 eviction_manager.go:189] "Eviction manager: starting control loop" May 20 18:59:03.041013 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.017099 2575 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 20 18:59:03.041013 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.017194 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 20 18:59:03.041013 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.017282 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 20 18:59:03.041013 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.017293 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 20 18:59:03.041013 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.017955 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 20 18:59:03.041013 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.017987 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-66.ec2.internal\" not found" May 20 18:59:03.083290 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.083255 2575 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 20 18:59:03.084405 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.084378 2575 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 20 18:59:03.084405 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.084406 2575 status_manager.go:230] "Starting to sync pod status with apiserver" May 20 18:59:03.084554 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.084431 2575 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 20 18:59:03.084554 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.084442 2575 kubelet.go:2451] "Starting kubelet main sync loop" May 20 18:59:03.084554 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.084489 2575 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 20 18:59:03.090265 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.090247 2575 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 20 18:59:03.117547 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.117515 2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 20 18:59:03.118440 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.118423 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasSufficientMemory" May 20 18:59:03.118533 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.118456 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasNoDiskPressure" May 20 18:59:03.118533 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.118471 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasSufficientPID" May 20 18:59:03.118533 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.118500 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-66.ec2.internal" May 20 18:59:03.124584 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.124570 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-66.ec2.internal" May 20 18:59:03.124643 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.124591 2575 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-66.ec2.internal\": node \"ip-10-0-138-66.ec2.internal\" not found" May 20 18:59:03.155325 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.155293 2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-66.ec2.internal\" not found" May 20 18:59:03.184705 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.184667 2575 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-66.ec2.internal"] May 20 18:59:03.184788 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.184765 2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 20 18:59:03.185711 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.185693 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasSufficientMemory" May 20 18:59:03.185802 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.185723 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasNoDiskPressure" May 20 18:59:03.185802 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.185733 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasSufficientPID" May 20 18:59:03.187961 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.187947 2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 20 18:59:03.188141 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.188125 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal" May 20 18:59:03.188227 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.188199 2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 20 18:59:03.188641 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.188627 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasSufficientMemory" May 20 18:59:03.188725 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.188651 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasNoDiskPressure" May 20 18:59:03.188725 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.188662 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasSufficientPID" May 20 18:59:03.188828 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.188751 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasSufficientMemory" May 20 18:59:03.188828 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.188781 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasNoDiskPressure" May 20 18:59:03.188828 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.188798 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasSufficientPID" May 20 18:59:03.190934 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.190916 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-66.ec2.internal" May 20 18:59:03.191026 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.190942 2575 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 20 18:59:03.191617 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.191603 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasSufficientMemory" May 20 18:59:03.191696 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.191624 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasNoDiskPressure" May 20 18:59:03.191696 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.191633 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeHasSufficientPID" May 20 18:59:03.220860 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.220840 2575 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-66.ec2.internal\" not found" node="ip-10-0-138-66.ec2.internal" May 20 18:59:03.225174 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.225159 2575 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-66.ec2.internal\" not found" node="ip-10-0-138-66.ec2.internal" May 20 18:59:03.255657 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.255625 2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-66.ec2.internal\" not found" May 20 18:59:03.256543 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.256521 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d991159dd187170fe148f8050e8b485-config\") pod \"kube-apiserver-proxy-ip-10-0-138-66.ec2.internal\" (UID: \"9d991159dd187170fe148f8050e8b485\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-66.ec2.internal" May 20 18:59:03.256629 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.256552 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/781b6dfbdd49af6160294e1b772d8d95-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal\" (UID: \"781b6dfbdd49af6160294e1b772d8d95\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal" May 20 18:59:03.256629 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.256573 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/781b6dfbdd49af6160294e1b772d8d95-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal\" (UID: \"781b6dfbdd49af6160294e1b772d8d95\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal" May 20 18:59:03.355925 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.355831 2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-66.ec2.internal\" not found" May 20 18:59:03.357023 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.356995 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/781b6dfbdd49af6160294e1b772d8d95-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal\" (UID: \"781b6dfbdd49af6160294e1b772d8d95\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal" May 20 18:59:03.357023 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.357020 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/781b6dfbdd49af6160294e1b772d8d95-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal\" (UID: \"781b6dfbdd49af6160294e1b772d8d95\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal" May 20 18:59:03.357188 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.357069 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/781b6dfbdd49af6160294e1b772d8d95-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal\" (UID: \"781b6dfbdd49af6160294e1b772d8d95\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal" May 20 18:59:03.357188 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.357092 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d991159dd187170fe148f8050e8b485-config\") pod \"kube-apiserver-proxy-ip-10-0-138-66.ec2.internal\" (UID: \"9d991159dd187170fe148f8050e8b485\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-66.ec2.internal" May 20 18:59:03.357188 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.357129 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9d991159dd187170fe148f8050e8b485-config\") pod \"kube-apiserver-proxy-ip-10-0-138-66.ec2.internal\" (UID: \"9d991159dd187170fe148f8050e8b485\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-66.ec2.internal" May 20 18:59:03.357188 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.357145 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/781b6dfbdd49af6160294e1b772d8d95-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal\" (UID: \"781b6dfbdd49af6160294e1b772d8d95\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal" May 20 18:59:03.456447 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.456413 2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-66.ec2.internal\" not found" May 20 18:59:03.523685 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.523653 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal" May 20 18:59:03.527526 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.527503 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-66.ec2.internal" May 20 18:59:03.557344 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.557309 2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-66.ec2.internal\" not found" May 20 18:59:03.657866 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.657780 2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-66.ec2.internal\" not found" May 20 18:59:03.758244 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.758198 2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-66.ec2.internal\" not found" May 20 18:59:03.850615 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.850583 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 20 18:59:03.851272 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.850716 2575 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 20 18:59:03.851272 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.850748 2575 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 20 18:59:03.858879 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.858856 2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-66.ec2.internal\" not found" May 20 18:59:03.952646 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.952557 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-19 18:54:02 +0000 UTC" deadline="2027-11-14 14:42:56.467091342 +0000 UTC" May 20 18:59:03.952646 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.952597 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13027h43m52.514499477s" May 20 18:59:03.955326 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.955308 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 20 18:59:03.959793 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:03.959772 2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-66.ec2.internal\" not found" May 20 18:59:03.970919 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.970896 2575 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 20 18:59:03.991552 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.991522 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hwt2d" May 20 18:59:03.998892 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:03.998874 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hwt2d" May 20 18:59:04.060517 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:04.060481 2575 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-66.ec2.internal\" not found" May 20 18:59:04.070251 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.070230 2575 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 20 18:59:04.124499 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:04.124466 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d991159dd187170fe148f8050e8b485.slice/crio-9124fa2c544e33bfe4365b4e4392655197f8a5cef19ebc4ce9f6c049561366e7 WatchSource:0}: Error finding container 9124fa2c544e33bfe4365b4e4392655197f8a5cef19ebc4ce9f6c049561366e7: Status 404 returned error can't find the container with id 9124fa2c544e33bfe4365b4e4392655197f8a5cef19ebc4ce9f6c049561366e7 May 20 18:59:04.124748 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:04.124733 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod781b6dfbdd49af6160294e1b772d8d95.slice/crio-f066b1e82187235cd3e3be0ac4a3480f3a45ef40d395e9e965868b68c144f807 WatchSource:0}: Error finding container f066b1e82187235cd3e3be0ac4a3480f3a45ef40d395e9e965868b68c144f807: Status 404 returned error can't find the container with id f066b1e82187235cd3e3be0ac4a3480f3a45ef40d395e9e965868b68c144f807 May 20 18:59:04.127977 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.127963 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 20 18:59:04.155531 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.155502 2575 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal" May 20 18:59:04.167917 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.167891 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 20 18:59:04.169344 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.169331 2575 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-66.ec2.internal" May 20 18:59:04.177126 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.177114 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 20 18:59:04.188598 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.188581 2575 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 20 18:59:04.734884 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.734842 2575 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 20 18:59:04.927393 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.927362 2575 apiserver.go:52] "Watching apiserver" May 20 18:59:04.936814 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.936787 2575 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 20 18:59:04.938700 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.938668 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-8bcz2","openshift-dns/node-resolver-gpvmh","openshift-image-registry/node-ca-d5z52","openshift-multus/network-metrics-daemon-84tr8","openshift-network-operator/iptables-alerter-wslfb","openshift-ovn-kubernetes/ovnkube-node-szh8j","kube-system/kube-apiserver-proxy-ip-10-0-138-66.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal","openshift-multus/multus-additional-cni-plugins-7fszs","openshift-multus/multus-xlhks","openshift-network-diagnostics/network-check-target-8hwbz","kube-system/konnectivity-agent-klzxc"] May 20 18:59:04.941430 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.941402 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:04.944112 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.944091 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 20 18:59:04.944112 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.944101 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xktxd\"" May 20 18:59:04.944295 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.944163 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 20 18:59:04.944471 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.944453 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 20 18:59:04.944576 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.944512 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 20 18:59:04.944631 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.944581 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 20 18:59:04.945854 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.945833 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gpvmh" May 20 18:59:04.947825 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.947777 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 20 18:59:04.948431 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.948174 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d5z52" May 20 18:59:04.948431 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.948223 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 20 18:59:04.948431 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.948243 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jdxgx\"" May 20 18:59:04.951089 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.950394 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 20 18:59:04.951089 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.950590 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wslfb" May 20 18:59:04.951089 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.950705 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 20 18:59:04.951089 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.950940 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 20 18:59:04.951468 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.951137 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sr9rq\"" May 20 18:59:04.953142 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.953059 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:04.953283 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:04.953179 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:04.953857 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.953837 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 20 18:59:04.953927 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.953860 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 20 18:59:04.954114 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.954085 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bwzk7\"" May 20 18:59:04.954320 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.954284 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 20 18:59:04.955477 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.955457 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.957793 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.957772 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 20 18:59:04.957951 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.957934 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:04.958230 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.958111 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jbtb6\"" May 20 18:59:04.959007 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.958530 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 20 18:59:04.959007 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.958932 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 20 18:59:04.960672 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.960650 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 20 18:59:04.960844 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.960826 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-l5ktb\"" May 20 18:59:04.961515 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.961497 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:04.961515 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.961512 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 20 18:59:04.961668 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.961654 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 20 18:59:04.962247 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.962229 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 20 18:59:04.963693 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.963670 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xlhks" May 20 18:59:04.963783 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.963745 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 20 18:59:04.963842 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.963823 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:04.963926 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:04.963902 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:04.964191 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964165 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-systemd-units\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.964191 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964217 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-run-netns\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.964191 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964258 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-node-log\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.964458 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964321 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.964458 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964435 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c634114-53bc-40fb-abff-081af0ff74eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:04.964549 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964482 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:04.964549 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964542 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8ctn\" (UniqueName: \"kubernetes.io/projected/8c283ba7-2436-4f29-b86c-97340c785bee-kube-api-access-k8ctn\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:04.964648 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964604 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c61a856-a85c-44a6-887d-7686c77f2eca-host-slash\") pod \"iptables-alerter-wslfb\" (UID: \"7c61a856-a85c-44a6-887d-7686c77f2eca\") " pod="openshift-network-operator/iptables-alerter-wslfb" May 20 18:59:04.964699 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964662 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4prdk\" (UniqueName: \"kubernetes.io/projected/23268351-bee9-4a36-937b-5d3eccdf2806-kube-api-access-4prdk\") pod \"node-resolver-gpvmh\" (UID: \"23268351-bee9-4a36-937b-5d3eccdf2806\") " pod="openshift-dns/node-resolver-gpvmh" May 20 18:59:04.964752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964704 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 20 18:59:04.964752 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964706 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-run-systemd\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.964874 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964849 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-var-lib-openvswitch\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.964945 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964907 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 20 18:59:04.964945 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964918 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-etc-openvswitch\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.965059 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964958 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 20 18:59:04.965059 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.964958 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8ckz\" (UniqueName: \"kubernetes.io/projected/1c634114-53bc-40fb-abff-081af0ff74eb-kube-api-access-p8ckz\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:04.965059 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.965000 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-c45f7\"" May 20 18:59:04.965059 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.965026 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/23268351-bee9-4a36-937b-5d3eccdf2806-hosts-file\") pod \"node-resolver-gpvmh\" (UID: \"23268351-bee9-4a36-937b-5d3eccdf2806\") " pod="openshift-dns/node-resolver-gpvmh" May 20 18:59:04.965302 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.965070 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-cni-netd\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.965302 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.965128 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b9b783e1-b420-4311-816e-61b4df8bb617-ovnkube-config\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.965302 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.965163 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b9b783e1-b420-4311-816e-61b4df8bb617-ovn-node-metrics-cert\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.965302 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.965198 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxltd\" (UniqueName: \"kubernetes.io/projected/b9b783e1-b420-4311-816e-61b4df8bb617-kube-api-access-lxltd\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.965302 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.965270 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:04.965512 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.965302 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-socket-dir\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:04.965512 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.965333 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c634114-53bc-40fb-abff-081af0ff74eb-system-cni-dir\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:04.965512 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.965362 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b9b783e1-b420-4311-816e-61b4df8bb617-ovnkube-script-lib\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.965512 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.965452 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-etc-selinux\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:04.965512 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.965487 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-sys-fs\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:04.966093 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966066 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kxd5\" (UniqueName: \"kubernetes.io/projected/21e22379-71df-4fbd-800e-e18398933ba7-kube-api-access-4kxd5\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:04.966174 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966127 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1c634114-53bc-40fb-abff-081af0ff74eb-os-release\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:04.966174 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966158 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1c634114-53bc-40fb-abff-081af0ff74eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:04.966304 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966183 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2e38623-90a4-414d-9517-087b938f83c1-host\") pod \"node-ca-d5z52\" (UID: \"f2e38623-90a4-414d-9517-087b938f83c1\") " pod="openshift-image-registry/node-ca-d5z52" May 20 18:59:04.966304 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966251 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-run-ovn-kubernetes\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.966304 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966283 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-device-dir\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:04.966304 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966287 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-scj7d\"" May 20 18:59:04.966490 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966309 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1c634114-53bc-40fb-abff-081af0ff74eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:04.966490 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966333 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj5gz\" (UniqueName: \"kubernetes.io/projected/f2e38623-90a4-414d-9517-087b938f83c1-kube-api-access-vj5gz\") pod \"node-ca-d5z52\" (UID: \"f2e38623-90a4-414d-9517-087b938f83c1\") " pod="openshift-image-registry/node-ca-d5z52" May 20 18:59:04.966490 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966355 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 20 18:59:04.966490 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966358 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c61a856-a85c-44a6-887d-7686c77f2eca-iptables-alerter-script\") pod \"iptables-alerter-wslfb\" (UID: \"7c61a856-a85c-44a6-887d-7686c77f2eca\") " pod="openshift-network-operator/iptables-alerter-wslfb" May 20 18:59:04.966490 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966383 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23268351-bee9-4a36-937b-5d3eccdf2806-tmp-dir\") pod \"node-resolver-gpvmh\" (UID: \"23268351-bee9-4a36-937b-5d3eccdf2806\") " pod="openshift-dns/node-resolver-gpvmh" May 20 18:59:04.966490 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966404 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-slash\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.966490 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966426 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-run-openvswitch\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.966490 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966448 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-registration-dir\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:04.966490 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966471 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f2e38623-90a4-414d-9517-087b938f83c1-serviceca\") pod \"node-ca-d5z52\" (UID: \"f2e38623-90a4-414d-9517-087b938f83c1\") " pod="openshift-image-registry/node-ca-d5z52" May 20 18:59:04.966490 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966492 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-kubelet\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.966980 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966514 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-run-ovn\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.966980 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966538 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b9b783e1-b420-4311-816e-61b4df8bb617-env-overrides\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.966980 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966562 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-log-socket\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.966980 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966584 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-cni-bin\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:04.966980 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966607 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c634114-53bc-40fb-abff-081af0ff74eb-cnibin\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:04.966980 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966631 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c634114-53bc-40fb-abff-081af0ff74eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:04.966980 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966654 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plcc6\" (UniqueName: \"kubernetes.io/projected/7c61a856-a85c-44a6-887d-7686c77f2eca-kube-api-access-plcc6\") pod \"iptables-alerter-wslfb\" (UID: \"7c61a856-a85c-44a6-887d-7686c77f2eca\") " pod="openshift-network-operator/iptables-alerter-wslfb" May 20 18:59:04.966980 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.966903 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-klzxc" May 20 18:59:04.969537 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.969301 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 20 18:59:04.969537 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.969312 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 20 18:59:04.969537 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.969304 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fknd8\"" May 20 18:59:04.999720 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.999671 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-19 18:54:03 +0000 UTC" deadline="2027-12-14 21:45:42.862360022 +0000 UTC" May 20 18:59:04.999720 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:04.999707 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13754h46m37.862656048s" May 20 18:59:05.056410 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.056381 2575 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 20 18:59:05.056808 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.056779 2575 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 20 18:59:05.066970 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.066944 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6c9126b-be8e-4c5f-b67b-940897245170-cni-binary-copy\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.067107 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.066979 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b9b783e1-b420-4311-816e-61b4df8bb617-ovnkube-script-lib\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.067107 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067004 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1c634114-53bc-40fb-abff-081af0ff74eb-os-release\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.067107 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067027 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-run-ovn-kubernetes\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.067107 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067081 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-run-ovn-kubernetes\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.067346 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067122 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-device-dir\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.067346 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067145 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-sysctl-d\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.067346 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067158 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1c634114-53bc-40fb-abff-081af0ff74eb-os-release\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.067346 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067165 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-systemd\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.067346 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067233 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-multus-socket-dir-parent\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.067346 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067238 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-device-dir\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.067346 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067267 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vj5gz\" (UniqueName: \"kubernetes.io/projected/f2e38623-90a4-414d-9517-087b938f83c1-kube-api-access-vj5gz\") pod \"node-ca-d5z52\" (UID: \"f2e38623-90a4-414d-9517-087b938f83c1\") " pod="openshift-image-registry/node-ca-d5z52" May 20 18:59:05.067346 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067289 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-slash\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.067346 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067316 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-registration-dir\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.067346 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067343 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-etc-kubernetes\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.067709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067354 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-slash\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.067709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067369 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f2e38623-90a4-414d-9517-087b938f83c1-serviceca\") pod \"node-ca-d5z52\" (UID: \"f2e38623-90a4-414d-9517-087b938f83c1\") " pod="openshift-image-registry/node-ca-d5z52" May 20 18:59:05.067709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067394 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-run-ovn\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.067709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067414 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-registration-dir\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.067709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067419 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-kubernetes\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.067709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067470 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-run-multus-certs\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.067709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067514 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-run-ovn\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.067709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067577 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c634114-53bc-40fb-abff-081af0ff74eb-cnibin\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.067709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067629 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c634114-53bc-40fb-abff-081af0ff74eb-cnibin\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.067709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067677 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-node-log\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.067709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067712 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-run-k8s-cni-cncf-io\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.068126 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067722 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b9b783e1-b420-4311-816e-61b4df8bb617-ovnkube-script-lib\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.068126 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067739 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c634114-53bc-40fb-abff-081af0ff74eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.068126 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067804 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-node-log\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.068126 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067839 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f2e38623-90a4-414d-9517-087b938f83c1-serviceca\") pod \"node-ca-d5z52\" (UID: \"f2e38623-90a4-414d-9517-087b938f83c1\") " pod="openshift-image-registry/node-ca-d5z52" May 20 18:59:05.068126 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067849 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:05.068126 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067889 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k8ctn\" (UniqueName: \"kubernetes.io/projected/8c283ba7-2436-4f29-b86c-97340c785bee-kube-api-access-k8ctn\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:05.068126 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:05.067933 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:05.068126 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:05.068004 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs podName:8c283ba7-2436-4f29-b86c-97340c785bee nodeName:}" failed. No retries permitted until 2026-05-20 18:59:05.567974558 +0000 UTC m=+3.113933406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs") pod "network-metrics-daemon-84tr8" (UID: "8c283ba7-2436-4f29-b86c-97340c785bee") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:05.068126 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.067917 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4prdk\" (UniqueName: \"kubernetes.io/projected/23268351-bee9-4a36-937b-5d3eccdf2806-kube-api-access-4prdk\") pod \"node-resolver-gpvmh\" (UID: \"23268351-bee9-4a36-937b-5d3eccdf2806\") " pod="openshift-dns/node-resolver-gpvmh" May 20 18:59:05.068126 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068123 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-etc-openvswitch\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068150 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b9b783e1-b420-4311-816e-61b4df8bb617-ovnkube-config\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068174 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lxltd\" (UniqueName: \"kubernetes.io/projected/b9b783e1-b420-4311-816e-61b4df8bb617-kube-api-access-lxltd\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068200 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068240 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/23268351-bee9-4a36-937b-5d3eccdf2806-hosts-file\") pod \"node-resolver-gpvmh\" (UID: \"23268351-bee9-4a36-937b-5d3eccdf2806\") " pod="openshift-dns/node-resolver-gpvmh" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068244 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-etc-openvswitch\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068248 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c634114-53bc-40fb-abff-081af0ff74eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068265 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-cni-netd\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068286 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-socket-dir\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068316 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-modprobe-d\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068332 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/23268351-bee9-4a36-937b-5d3eccdf2806-hosts-file\") pod \"node-resolver-gpvmh\" (UID: \"23268351-bee9-4a36-937b-5d3eccdf2806\") " pod="openshift-dns/node-resolver-gpvmh" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068336 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-cni-netd\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068338 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-sysctl-conf\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068388 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-host\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068379 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068426 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-cnibin\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068433 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-socket-dir\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.068560 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068451 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-run-netns\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068480 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c634114-53bc-40fb-abff-081af0ff74eb-system-cni-dir\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068503 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-etc-selinux\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068528 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4kxd5\" (UniqueName: \"kubernetes.io/projected/21e22379-71df-4fbd-800e-e18398933ba7-kube-api-access-4kxd5\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068553 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-sys\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068554 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c634114-53bc-40fb-abff-081af0ff74eb-system-cni-dir\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068568 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-etc-selinux\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068577 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-lib-modules\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068604 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-tuned\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068628 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-var-lib-cni-multus\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068656 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1c634114-53bc-40fb-abff-081af0ff74eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068686 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2e38623-90a4-414d-9517-087b938f83c1-host\") pod \"node-ca-d5z52\" (UID: \"f2e38623-90a4-414d-9517-087b938f83c1\") " pod="openshift-image-registry/node-ca-d5z52" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068737 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2e38623-90a4-414d-9517-087b938f83c1-host\") pod \"node-ca-d5z52\" (UID: \"f2e38623-90a4-414d-9517-087b938f83c1\") " pod="openshift-image-registry/node-ca-d5z52" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068774 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-var-lib-kubelet\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068820 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-var-lib-kubelet\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068849 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-multus-conf-dir\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068867 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b9b783e1-b420-4311-816e-61b4df8bb617-ovnkube-config\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.069418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068875 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a6c9126b-be8e-4c5f-b67b-940897245170-multus-daemon-config\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.070155 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068900 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxbbk\" (UniqueName: \"kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk\") pod \"network-check-target-8hwbz\" (UID: \"44861413-0803-43a6-9a94-e2e0085120c6\") " pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:05.070155 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068937 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1c634114-53bc-40fb-abff-081af0ff74eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.070155 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068966 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c61a856-a85c-44a6-887d-7686c77f2eca-iptables-alerter-script\") pod \"iptables-alerter-wslfb\" (UID: \"7c61a856-a85c-44a6-887d-7686c77f2eca\") " pod="openshift-network-operator/iptables-alerter-wslfb" May 20 18:59:05.070155 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.068997 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23268351-bee9-4a36-937b-5d3eccdf2806-tmp-dir\") pod \"node-resolver-gpvmh\" (UID: \"23268351-bee9-4a36-937b-5d3eccdf2806\") " pod="openshift-dns/node-resolver-gpvmh" May 20 18:59:05.070155 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.069022 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-run-openvswitch\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.070155 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.069047 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-multus-cni-dir\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.070155 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070107 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1c634114-53bc-40fb-abff-081af0ff74eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.070497 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070187 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-kubelet\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.070497 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070247 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b9b783e1-b420-4311-816e-61b4df8bb617-env-overrides\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.070497 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070281 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c61a856-a85c-44a6-887d-7686c77f2eca-iptables-alerter-script\") pod \"iptables-alerter-wslfb\" (UID: \"7c61a856-a85c-44a6-887d-7686c77f2eca\") " pod="openshift-network-operator/iptables-alerter-wslfb" May 20 18:59:05.070497 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070343 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-kubelet\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.070497 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070288 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-os-release\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.070497 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070395 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-hostroot\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.070497 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070403 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-run-openvswitch\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.070818 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070651 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1c634114-53bc-40fb-abff-081af0ff74eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.070818 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070719 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23268351-bee9-4a36-937b-5d3eccdf2806-tmp-dir\") pod \"node-resolver-gpvmh\" (UID: \"23268351-bee9-4a36-937b-5d3eccdf2806\") " pod="openshift-dns/node-resolver-gpvmh" May 20 18:59:05.070818 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070795 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8bdc28d-e9ef-4ce6-9a83-1947176e82a4-agent-certs\") pod \"konnectivity-agent-klzxc\" (UID: \"e8bdc28d-e9ef-4ce6-9a83-1947176e82a4\") " pod="kube-system/konnectivity-agent-klzxc" May 20 18:59:05.070954 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070821 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b9b783e1-b420-4311-816e-61b4df8bb617-env-overrides\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.070954 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070854 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-log-socket\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.070954 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070884 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-cni-bin\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.070954 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070918 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-system-cni-dir\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.071118 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070951 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-var-lib-cni-bin\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.071118 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070980 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-log-socket\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.071118 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.070986 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tg88\" (UniqueName: \"kubernetes.io/projected/a6c9126b-be8e-4c5f-b67b-940897245170-kube-api-access-9tg88\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.071118 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071038 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8bdc28d-e9ef-4ce6-9a83-1947176e82a4-konnectivity-ca\") pod \"konnectivity-agent-klzxc\" (UID: \"e8bdc28d-e9ef-4ce6-9a83-1947176e82a4\") " pod="kube-system/konnectivity-agent-klzxc" May 20 18:59:05.071118 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071038 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-cni-bin\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.071118 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071098 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-run\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.071397 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071135 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c634114-53bc-40fb-abff-081af0ff74eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.071397 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071170 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-plcc6\" (UniqueName: \"kubernetes.io/projected/7c61a856-a85c-44a6-887d-7686c77f2eca-kube-api-access-plcc6\") pod \"iptables-alerter-wslfb\" (UID: \"7c61a856-a85c-44a6-887d-7686c77f2eca\") " pod="openshift-network-operator/iptables-alerter-wslfb" May 20 18:59:05.071397 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071196 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-systemd-units\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.071397 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071254 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-run-netns\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.071397 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071287 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.071397 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071321 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-sys-fs\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.071397 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071355 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gkz2\" (UniqueName: \"kubernetes.io/projected/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-kube-api-access-8gkz2\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.071397 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071383 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c61a856-a85c-44a6-887d-7686c77f2eca-host-slash\") pod \"iptables-alerter-wslfb\" (UID: \"7c61a856-a85c-44a6-887d-7686c77f2eca\") " pod="openshift-network-operator/iptables-alerter-wslfb" May 20 18:59:05.071709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071420 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-run-systemd\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.071709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071457 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-var-lib-openvswitch\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.071709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071491 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-tmp\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.071709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071525 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p8ckz\" (UniqueName: \"kubernetes.io/projected/1c634114-53bc-40fb-abff-081af0ff74eb-kube-api-access-p8ckz\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.071709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071557 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b9b783e1-b420-4311-816e-61b4df8bb617-ovn-node-metrics-cert\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.071709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071584 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-sysconfig\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.071709 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071634 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.071986 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071720 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/21e22379-71df-4fbd-800e-e18398933ba7-sys-fs\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.071986 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071796 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-host-run-netns\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.071986 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071871 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-systemd-units\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.071986 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071923 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c61a856-a85c-44a6-887d-7686c77f2eca-host-slash\") pod \"iptables-alerter-wslfb\" (UID: \"7c61a856-a85c-44a6-887d-7686c77f2eca\") " pod="openshift-network-operator/iptables-alerter-wslfb" May 20 18:59:05.071986 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.071965 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-run-systemd\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.072322 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.072299 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 20 18:59:05.073888 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.072976 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c634114-53bc-40fb-abff-081af0ff74eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.073888 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.073048 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9b783e1-b420-4311-816e-61b4df8bb617-var-lib-openvswitch\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.075714 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.075517 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b9b783e1-b420-4311-816e-61b4df8bb617-ovn-node-metrics-cert\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.079528 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.079410 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kxd5\" (UniqueName: \"kubernetes.io/projected/21e22379-71df-4fbd-800e-e18398933ba7-kube-api-access-4kxd5\") pod \"aws-ebs-csi-driver-node-v9jpb\" (UID: \"21e22379-71df-4fbd-800e-e18398933ba7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.079528 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.079482 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8ctn\" (UniqueName: \"kubernetes.io/projected/8c283ba7-2436-4f29-b86c-97340c785bee-kube-api-access-k8ctn\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:05.079528 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.079519 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj5gz\" (UniqueName: \"kubernetes.io/projected/f2e38623-90a4-414d-9517-087b938f83c1-kube-api-access-vj5gz\") pod \"node-ca-d5z52\" (UID: \"f2e38623-90a4-414d-9517-087b938f83c1\") " pod="openshift-image-registry/node-ca-d5z52" May 20 18:59:05.079714 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.079633 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4prdk\" (UniqueName: \"kubernetes.io/projected/23268351-bee9-4a36-937b-5d3eccdf2806-kube-api-access-4prdk\") pod \"node-resolver-gpvmh\" (UID: \"23268351-bee9-4a36-937b-5d3eccdf2806\") " pod="openshift-dns/node-resolver-gpvmh" May 20 18:59:05.080689 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.080660 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxltd\" (UniqueName: \"kubernetes.io/projected/b9b783e1-b420-4311-816e-61b4df8bb617-kube-api-access-lxltd\") pod \"ovnkube-node-szh8j\" (UID: \"b9b783e1-b420-4311-816e-61b4df8bb617\") " pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.081464 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.081443 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plcc6\" (UniqueName: \"kubernetes.io/projected/7c61a856-a85c-44a6-887d-7686c77f2eca-kube-api-access-plcc6\") pod \"iptables-alerter-wslfb\" (UID: \"7c61a856-a85c-44a6-887d-7686c77f2eca\") " pod="openshift-network-operator/iptables-alerter-wslfb" May 20 18:59:05.082014 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.081990 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8ckz\" (UniqueName: \"kubernetes.io/projected/1c634114-53bc-40fb-abff-081af0ff74eb-kube-api-access-p8ckz\") pod \"multus-additional-cni-plugins-7fszs\" (UID: \"1c634114-53bc-40fb-abff-081af0ff74eb\") " pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.089295 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.089250 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal" event={"ID":"781b6dfbdd49af6160294e1b772d8d95","Type":"ContainerStarted","Data":"f066b1e82187235cd3e3be0ac4a3480f3a45ef40d395e9e965868b68c144f807"} May 20 18:59:05.090309 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.090268 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-66.ec2.internal" event={"ID":"9d991159dd187170fe148f8050e8b485","Type":"ContainerStarted","Data":"9124fa2c544e33bfe4365b4e4392655197f8a5cef19ebc4ce9f6c049561366e7"} May 20 18:59:05.172306 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172267 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-modprobe-d\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.172306 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172309 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-sysctl-conf\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.172530 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172332 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-host\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.172530 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172352 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-cnibin\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.172530 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172373 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-run-netns\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.172530 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172422 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-run-netns\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.172530 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172422 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-modprobe-d\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.172530 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172436 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-host\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.172530 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172445 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-cnibin\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.172530 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172461 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-sys\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.172530 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172485 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-lib-modules\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.172530 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172519 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-sysctl-conf\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172542 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-sys\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172579 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-tuned\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172587 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-lib-modules\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172604 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-var-lib-cni-multus\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172648 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-var-lib-cni-multus\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172735 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-var-lib-kubelet\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172768 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-var-lib-kubelet\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172793 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-multus-conf-dir\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172818 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a6c9126b-be8e-4c5f-b67b-940897245170-multus-daemon-config\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172833 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-var-lib-kubelet\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172845 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-var-lib-kubelet\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172869 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-multus-conf-dir\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172845 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kxbbk\" (UniqueName: \"kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk\") pod \"network-check-target-8hwbz\" (UID: \"44861413-0803-43a6-9a94-e2e0085120c6\") " pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172915 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-multus-cni-dir\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.172955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.172940 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-os-release\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173015 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-multus-cni-dir\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173052 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-os-release\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173078 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-hostroot\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173095 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8bdc28d-e9ef-4ce6-9a83-1947176e82a4-agent-certs\") pod \"konnectivity-agent-klzxc\" (UID: \"e8bdc28d-e9ef-4ce6-9a83-1947176e82a4\") " pod="kube-system/konnectivity-agent-klzxc" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173135 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-system-cni-dir\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173143 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-hostroot\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173160 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-var-lib-cni-bin\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173199 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-var-lib-cni-bin\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173227 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-system-cni-dir\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173312 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9tg88\" (UniqueName: \"kubernetes.io/projected/a6c9126b-be8e-4c5f-b67b-940897245170-kube-api-access-9tg88\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173338 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8bdc28d-e9ef-4ce6-9a83-1947176e82a4-konnectivity-ca\") pod \"konnectivity-agent-klzxc\" (UID: \"e8bdc28d-e9ef-4ce6-9a83-1947176e82a4\") " pod="kube-system/konnectivity-agent-klzxc" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173356 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-run\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173380 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8gkz2\" (UniqueName: \"kubernetes.io/projected/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-kube-api-access-8gkz2\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173401 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-tmp\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173417 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-sysconfig\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173447 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6c9126b-be8e-4c5f-b67b-940897245170-cni-binary-copy\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173451 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-run\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.173660 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173461 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a6c9126b-be8e-4c5f-b67b-940897245170-multus-daemon-config\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173476 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-sysctl-d\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173502 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-systemd\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173525 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-multus-socket-dir-parent\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173560 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-etc-kubernetes\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173564 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-sysconfig\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173587 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-kubernetes\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173589 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-sysctl-d\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173612 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-run-multus-certs\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173624 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-multus-socket-dir-parent\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173643 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-run-k8s-cni-cncf-io\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173707 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-run-k8s-cni-cncf-io\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173749 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-etc-kubernetes\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173560 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-systemd\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173829 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-kubernetes\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173871 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a6c9126b-be8e-4c5f-b67b-940897245170-host-run-multus-certs\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.173972 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8bdc28d-e9ef-4ce6-9a83-1947176e82a4-konnectivity-ca\") pod \"konnectivity-agent-klzxc\" (UID: \"e8bdc28d-e9ef-4ce6-9a83-1947176e82a4\") " pod="kube-system/konnectivity-agent-klzxc" May 20 18:59:05.174443 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.174024 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6c9126b-be8e-4c5f-b67b-940897245170-cni-binary-copy\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.175379 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.175354 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-etc-tuned\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.175792 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.175776 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8bdc28d-e9ef-4ce6-9a83-1947176e82a4-agent-certs\") pod \"konnectivity-agent-klzxc\" (UID: \"e8bdc28d-e9ef-4ce6-9a83-1947176e82a4\") " pod="kube-system/konnectivity-agent-klzxc" May 20 18:59:05.175792 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.175787 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-tmp\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.179160 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:05.179127 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 18:59:05.179160 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:05.179149 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 18:59:05.179160 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:05.179162 2575 projected.go:194] Error preparing data for projected volume kube-api-access-kxbbk for pod openshift-network-diagnostics/network-check-target-8hwbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:05.179367 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:05.179245 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk podName:44861413-0803-43a6-9a94-e2e0085120c6 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:05.67922574 +0000 UTC m=+3.225184586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kxbbk" (UniqueName: "kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk") pod "network-check-target-8hwbz" (UID: "44861413-0803-43a6-9a94-e2e0085120c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:05.181581 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.181559 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tg88\" (UniqueName: \"kubernetes.io/projected/a6c9126b-be8e-4c5f-b67b-940897245170-kube-api-access-9tg88\") pod \"multus-xlhks\" (UID: \"a6c9126b-be8e-4c5f-b67b-940897245170\") " pod="openshift-multus/multus-xlhks" May 20 18:59:05.181697 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.181591 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gkz2\" (UniqueName: \"kubernetes.io/projected/f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a-kube-api-access-8gkz2\") pod \"tuned-8bcz2\" (UID: \"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a\") " pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.255854 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.255738 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7fszs" May 20 18:59:05.271724 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.271685 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gpvmh" May 20 18:59:05.279401 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.279379 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d5z52" May 20 18:59:05.285966 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.285944 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wslfb" May 20 18:59:05.293537 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.293522 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:05.301077 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.301059 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" May 20 18:59:05.307596 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.307578 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" May 20 18:59:05.315144 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.315121 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xlhks" May 20 18:59:05.320651 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.320631 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-klzxc" May 20 18:59:05.576673 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.576589 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:05.576843 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:05.576759 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:05.576843 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:05.576836 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs podName:8c283ba7-2436-4f29-b86c-97340c785bee nodeName:}" failed. No retries permitted until 2026-05-20 18:59:06.576813278 +0000 UTC m=+4.122772129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs") pod "network-metrics-daemon-84tr8" (UID: "8c283ba7-2436-4f29-b86c-97340c785bee") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:05.778667 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:05.778637 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kxbbk\" (UniqueName: \"kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk\") pod \"network-check-target-8hwbz\" (UID: \"44861413-0803-43a6-9a94-e2e0085120c6\") " pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:05.778805 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:05.778764 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 18:59:05.778805 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:05.778781 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 18:59:05.778805 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:05.778790 2575 projected.go:194] Error preparing data for projected volume kube-api-access-kxbbk for pod openshift-network-diagnostics/network-check-target-8hwbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:05.778898 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:05.778839 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk podName:44861413-0803-43a6-9a94-e2e0085120c6 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:06.778826002 +0000 UTC m=+4.324784835 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxbbk" (UniqueName: "kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk") pod "network-check-target-8hwbz" (UID: "44861413-0803-43a6-9a94-e2e0085120c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:05.825392 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:05.825364 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c634114_53bc_40fb_abff_081af0ff74eb.slice/crio-df368a42e618a5af4704dfd834404daf852fd488421bdfae758a29caae6dfa72 WatchSource:0}: Error finding container df368a42e618a5af4704dfd834404daf852fd488421bdfae758a29caae6dfa72: Status 404 returned error can't find the container with id df368a42e618a5af4704dfd834404daf852fd488421bdfae758a29caae6dfa72 May 20 18:59:05.825646 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:05.825624 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e38623_90a4_414d_9517_087b938f83c1.slice/crio-9e8a76c872738cb75eea26a08f20c1798cc446375a88823ddfb69a265bb98033 WatchSource:0}: Error finding container 9e8a76c872738cb75eea26a08f20c1798cc446375a88823ddfb69a265bb98033: Status 404 returned error can't find the container with id 9e8a76c872738cb75eea26a08f20c1798cc446375a88823ddfb69a265bb98033 May 20 18:59:05.826297 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:05.826189 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9b783e1_b420_4311_816e_61b4df8bb617.slice/crio-04be137221b7802913b6be031fd6d5536454491d3138e8c593f86812d4f0b2fe WatchSource:0}: Error finding container 04be137221b7802913b6be031fd6d5536454491d3138e8c593f86812d4f0b2fe: Status 404 returned error can't find the container with id 04be137221b7802913b6be031fd6d5536454491d3138e8c593f86812d4f0b2fe May 20 18:59:05.828710 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:05.828688 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c61a856_a85c_44a6_887d_7686c77f2eca.slice/crio-10f5cdd70d8b67ba50aef9ed68e63284a5046a35a6cf078788c1473548cdd936 WatchSource:0}: Error finding container 10f5cdd70d8b67ba50aef9ed68e63284a5046a35a6cf078788c1473548cdd936: Status 404 returned error can't find the container with id 10f5cdd70d8b67ba50aef9ed68e63284a5046a35a6cf078788c1473548cdd936 May 20 18:59:05.829630 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:05.829612 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6c9126b_be8e_4c5f_b67b_940897245170.slice/crio-12689abbfb057eb58f937e48d3b5bb1724fd0017fb1b2792fd3add2c316d00a7 WatchSource:0}: Error finding container 12689abbfb057eb58f937e48d3b5bb1724fd0017fb1b2792fd3add2c316d00a7: Status 404 returned error can't find the container with id 12689abbfb057eb58f937e48d3b5bb1724fd0017fb1b2792fd3add2c316d00a7 May 20 18:59:05.830549 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:05.830526 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23268351_bee9_4a36_937b_5d3eccdf2806.slice/crio-6968e3f314bd2f1f1879c2d766ce2992f32f59ae0f0f26a7f718a88b5acabd0b WatchSource:0}: Error finding container 6968e3f314bd2f1f1879c2d766ce2992f32f59ae0f0f26a7f718a88b5acabd0b: Status 404 returned error can't find the container with id 6968e3f314bd2f1f1879c2d766ce2992f32f59ae0f0f26a7f718a88b5acabd0b May 20 18:59:05.831447 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:05.831427 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f3ea7a_d6fb_4eb4_ace8_a22729fed55a.slice/crio-32d76e6eca5b84c307d4c80caf49d346244efa0b04a7e05066bb1c76ac5c0027 WatchSource:0}: Error finding container 32d76e6eca5b84c307d4c80caf49d346244efa0b04a7e05066bb1c76ac5c0027: Status 404 returned error can't find the container with id 32d76e6eca5b84c307d4c80caf49d346244efa0b04a7e05066bb1c76ac5c0027 May 20 18:59:05.834488 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:05.834328 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e22379_71df_4fbd_800e_e18398933ba7.slice/crio-57be35c34e01d49e7d560114907391950ed68b597dfe8c702ad707d811957213 WatchSource:0}: Error finding container 57be35c34e01d49e7d560114907391950ed68b597dfe8c702ad707d811957213: Status 404 returned error can't find the container with id 57be35c34e01d49e7d560114907391950ed68b597dfe8c702ad707d811957213 May 20 18:59:05.834857 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:05.834832 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8bdc28d_e9ef_4ce6_9a83_1947176e82a4.slice/crio-0eebd052a124d356d96e5e23dbaa498b4311b000d3e1c3dde0d3edba70b4b535 WatchSource:0}: Error finding container 0eebd052a124d356d96e5e23dbaa498b4311b000d3e1c3dde0d3edba70b4b535: Status 404 returned error can't find the container with id 0eebd052a124d356d96e5e23dbaa498b4311b000d3e1c3dde0d3edba70b4b535 May 20 18:59:06.000714 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.000667 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-19 18:54:03 +0000 UTC" deadline="2027-10-20 22:35:45.701735569 +0000 UTC" May 20 18:59:06.000714 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.000710 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12435h36m39.701030555s" May 20 18:59:06.085383 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.085358 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:06.085504 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:06.085483 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:06.092682 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.092654 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d5z52" event={"ID":"f2e38623-90a4-414d-9517-087b938f83c1","Type":"ContainerStarted","Data":"9e8a76c872738cb75eea26a08f20c1798cc446375a88823ddfb69a265bb98033"} May 20 18:59:06.093627 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.093605 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xlhks" event={"ID":"a6c9126b-be8e-4c5f-b67b-940897245170","Type":"ContainerStarted","Data":"12689abbfb057eb58f937e48d3b5bb1724fd0017fb1b2792fd3add2c316d00a7"} May 20 18:59:06.094874 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.094856 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-klzxc" event={"ID":"e8bdc28d-e9ef-4ce6-9a83-1947176e82a4","Type":"ContainerStarted","Data":"0eebd052a124d356d96e5e23dbaa498b4311b000d3e1c3dde0d3edba70b4b535"} May 20 18:59:06.095675 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.095656 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gpvmh" event={"ID":"23268351-bee9-4a36-937b-5d3eccdf2806","Type":"ContainerStarted","Data":"6968e3f314bd2f1f1879c2d766ce2992f32f59ae0f0f26a7f718a88b5acabd0b"} May 20 18:59:06.096532 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.096513 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7fszs" event={"ID":"1c634114-53bc-40fb-abff-081af0ff74eb","Type":"ContainerStarted","Data":"df368a42e618a5af4704dfd834404daf852fd488421bdfae758a29caae6dfa72"} May 20 18:59:06.097975 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.097957 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-66.ec2.internal" event={"ID":"9d991159dd187170fe148f8050e8b485","Type":"ContainerStarted","Data":"d5fe2e5c69ec7d8d090b13daedc4e0a41c2360cd6234eaa609fb3be339f9d1e1"} May 20 18:59:06.098937 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.098919 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" event={"ID":"21e22379-71df-4fbd-800e-e18398933ba7","Type":"ContainerStarted","Data":"57be35c34e01d49e7d560114907391950ed68b597dfe8c702ad707d811957213"} May 20 18:59:06.099835 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.099815 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" event={"ID":"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a","Type":"ContainerStarted","Data":"32d76e6eca5b84c307d4c80caf49d346244efa0b04a7e05066bb1c76ac5c0027"} May 20 18:59:06.100641 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.100623 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wslfb" event={"ID":"7c61a856-a85c-44a6-887d-7686c77f2eca","Type":"ContainerStarted","Data":"10f5cdd70d8b67ba50aef9ed68e63284a5046a35a6cf078788c1473548cdd936"} May 20 18:59:06.101412 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.101389 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" event={"ID":"b9b783e1-b420-4311-816e-61b4df8bb617","Type":"ContainerStarted","Data":"04be137221b7802913b6be031fd6d5536454491d3138e8c593f86812d4f0b2fe"} May 20 18:59:06.113312 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.113272 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-66.ec2.internal" podStartSLOduration=2.113260229 podStartE2EDuration="2.113260229s" podCreationTimestamp="2026-05-20 18:59:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 18:59:06.112847969 +0000 UTC m=+3.658806823" watchObservedRunningTime="2026-05-20 18:59:06.113260229 +0000 UTC m=+3.659219085" May 20 18:59:06.586665 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.586626 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:06.586848 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:06.586813 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:06.586927 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:06.586894 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs podName:8c283ba7-2436-4f29-b86c-97340c785bee nodeName:}" failed. No retries permitted until 2026-05-20 18:59:08.586873999 +0000 UTC m=+6.132832837 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs") pod "network-metrics-daemon-84tr8" (UID: "8c283ba7-2436-4f29-b86c-97340c785bee") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:06.789896 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:06.789155 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kxbbk\" (UniqueName: \"kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk\") pod \"network-check-target-8hwbz\" (UID: \"44861413-0803-43a6-9a94-e2e0085120c6\") " pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:06.789896 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:06.789389 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 18:59:06.789896 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:06.789412 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 18:59:06.789896 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:06.789425 2575 projected.go:194] Error preparing data for projected volume kube-api-access-kxbbk for pod openshift-network-diagnostics/network-check-target-8hwbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:06.789896 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:06.789485 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk podName:44861413-0803-43a6-9a94-e2e0085120c6 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:08.789466984 +0000 UTC m=+6.335425824 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxbbk" (UniqueName: "kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk") pod "network-check-target-8hwbz" (UID: "44861413-0803-43a6-9a94-e2e0085120c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:07.088310 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:07.087702 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:07.088310 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:07.087837 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:07.113798 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:07.113761 2575 generic.go:358] "Generic (PLEG): container finished" podID="781b6dfbdd49af6160294e1b772d8d95" containerID="f696d6987947be5420f3e283119a100d04dfb79a0c3ce6e12084ad552384f9cc" exitCode=0 May 20 18:59:07.114944 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:07.114864 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal" event={"ID":"781b6dfbdd49af6160294e1b772d8d95","Type":"ContainerDied","Data":"f696d6987947be5420f3e283119a100d04dfb79a0c3ce6e12084ad552384f9cc"} May 20 18:59:07.724646 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:07.724565 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-j9r94"] May 20 18:59:07.727561 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:07.727530 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:07.727688 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:07.727611 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:07.799263 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:07.796472 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1b20dd7a-c2dd-489e-bd5e-28916190af81-kubelet-config\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:07.799263 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:07.796530 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1b20dd7a-c2dd-489e-bd5e-28916190af81-dbus\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:07.799263 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:07.796560 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:07.897303 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:07.897264 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1b20dd7a-c2dd-489e-bd5e-28916190af81-kubelet-config\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:07.897476 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:07.897335 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1b20dd7a-c2dd-489e-bd5e-28916190af81-dbus\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:07.897476 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:07.897367 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:07.897600 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:07.897490 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 20 18:59:07.897600 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:07.897552 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret podName:1b20dd7a-c2dd-489e-bd5e-28916190af81 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:08.397534231 +0000 UTC m=+5.943493069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret") pod "global-pull-secret-syncer-j9r94" (UID: "1b20dd7a-c2dd-489e-bd5e-28916190af81") : object "kube-system"/"original-pull-secret" not registered May 20 18:59:07.897832 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:07.897810 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1b20dd7a-c2dd-489e-bd5e-28916190af81-kubelet-config\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:07.897965 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:07.897949 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1b20dd7a-c2dd-489e-bd5e-28916190af81-dbus\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:08.085192 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:08.085100 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:08.085369 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:08.085271 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:08.126529 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:08.126492 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal" event={"ID":"781b6dfbdd49af6160294e1b772d8d95","Type":"ContainerStarted","Data":"970df459a3c559aa03b5fcda257f25f0fbaf3a03d9866fefc988fc12cec03f0c"} May 20 18:59:08.407125 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:08.407036 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:08.407310 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:08.407196 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 20 18:59:08.407310 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:08.407291 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret podName:1b20dd7a-c2dd-489e-bd5e-28916190af81 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:09.407269422 +0000 UTC m=+6.953228259 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret") pod "global-pull-secret-syncer-j9r94" (UID: "1b20dd7a-c2dd-489e-bd5e-28916190af81") : object "kube-system"/"original-pull-secret" not registered May 20 18:59:08.608880 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:08.608843 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:08.609048 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:08.609016 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:08.609126 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:08.609082 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs podName:8c283ba7-2436-4f29-b86c-97340c785bee nodeName:}" failed. No retries permitted until 2026-05-20 18:59:12.609064539 +0000 UTC m=+10.155023391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs") pod "network-metrics-daemon-84tr8" (UID: "8c283ba7-2436-4f29-b86c-97340c785bee") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:08.810698 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:08.810583 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kxbbk\" (UniqueName: \"kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk\") pod \"network-check-target-8hwbz\" (UID: \"44861413-0803-43a6-9a94-e2e0085120c6\") " pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:08.810856 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:08.810768 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 18:59:08.810856 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:08.810791 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 18:59:08.810856 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:08.810804 2575 projected.go:194] Error preparing data for projected volume kube-api-access-kxbbk for pod openshift-network-diagnostics/network-check-target-8hwbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:08.811011 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:08.810868 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk podName:44861413-0803-43a6-9a94-e2e0085120c6 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:12.810849131 +0000 UTC m=+10.356807985 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxbbk" (UniqueName: "kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk") pod "network-check-target-8hwbz" (UID: "44861413-0803-43a6-9a94-e2e0085120c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:09.085779 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:09.085636 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:09.085939 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:09.085776 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:09.086262 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:09.085636 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:09.086262 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:09.086161 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:09.416633 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:09.416547 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:09.417096 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:09.416714 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 20 18:59:09.417096 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:09.416779 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret podName:1b20dd7a-c2dd-489e-bd5e-28916190af81 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:11.416760645 +0000 UTC m=+8.962719481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret") pod "global-pull-secret-syncer-j9r94" (UID: "1b20dd7a-c2dd-489e-bd5e-28916190af81") : object "kube-system"/"original-pull-secret" not registered May 20 18:59:10.084929 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:10.084889 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:10.085129 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:10.085024 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:11.085548 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:11.085280 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:11.085548 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:11.085280 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:11.085548 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:11.085416 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:11.085548 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:11.085503 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:11.435536 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:11.435335 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:11.435536 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:11.435469 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 20 18:59:11.435536 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:11.435536 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret podName:1b20dd7a-c2dd-489e-bd5e-28916190af81 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:15.435518868 +0000 UTC m=+12.981477706 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret") pod "global-pull-secret-syncer-j9r94" (UID: "1b20dd7a-c2dd-489e-bd5e-28916190af81") : object "kube-system"/"original-pull-secret" not registered May 20 18:59:12.085156 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:12.084948 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:12.085156 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:12.085096 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:12.645725 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:12.645683 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:12.646173 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:12.645863 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:12.646173 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:12.645938 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs podName:8c283ba7-2436-4f29-b86c-97340c785bee nodeName:}" failed. No retries permitted until 2026-05-20 18:59:20.64591401 +0000 UTC m=+18.191872848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs") pod "network-metrics-daemon-84tr8" (UID: "8c283ba7-2436-4f29-b86c-97340c785bee") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:12.847264 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:12.847182 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kxbbk\" (UniqueName: \"kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk\") pod \"network-check-target-8hwbz\" (UID: \"44861413-0803-43a6-9a94-e2e0085120c6\") " pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:12.847446 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:12.847364 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 18:59:12.847446 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:12.847388 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 18:59:12.847446 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:12.847402 2575 projected.go:194] Error preparing data for projected volume kube-api-access-kxbbk for pod openshift-network-diagnostics/network-check-target-8hwbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:12.847605 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:12.847462 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk podName:44861413-0803-43a6-9a94-e2e0085120c6 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:20.847444034 +0000 UTC m=+18.393402870 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxbbk" (UniqueName: "kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk") pod "network-check-target-8hwbz" (UID: "44861413-0803-43a6-9a94-e2e0085120c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:13.087372 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:13.086627 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:13.087372 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:13.086659 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:13.087372 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:13.086764 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:13.087372 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:13.087280 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:14.085060 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:14.085017 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:14.085520 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:14.085159 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:15.085567 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:15.085532 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:15.086014 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:15.085589 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:15.086014 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:15.085676 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:15.086121 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:15.086039 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:15.468850 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:15.468754 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:15.469010 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:15.468913 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 20 18:59:15.469010 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:15.468976 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret podName:1b20dd7a-c2dd-489e-bd5e-28916190af81 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:23.468958761 +0000 UTC m=+21.014917593 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret") pod "global-pull-secret-syncer-j9r94" (UID: "1b20dd7a-c2dd-489e-bd5e-28916190af81") : object "kube-system"/"original-pull-secret" not registered May 20 18:59:16.085217 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:16.085171 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:16.085397 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:16.085296 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:17.085519 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:17.085487 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:17.085962 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:17.085496 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:17.085962 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:17.085594 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:17.085962 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:17.085699 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:18.084625 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:18.084589 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:18.084790 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:18.084708 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:19.085311 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:19.085273 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:19.085778 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:19.085273 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:19.085778 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:19.085413 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:19.085778 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:19.085518 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:20.085499 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:20.085427 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:20.085883 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:20.085543 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:20.708000 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:20.707962 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:20.708187 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:20.708115 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:20.708187 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:20.708178 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs podName:8c283ba7-2436-4f29-b86c-97340c785bee nodeName:}" failed. No retries permitted until 2026-05-20 18:59:36.708158613 +0000 UTC m=+34.254117460 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs") pod "network-metrics-daemon-84tr8" (UID: "8c283ba7-2436-4f29-b86c-97340c785bee") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:20.909730 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:20.909686 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kxbbk\" (UniqueName: \"kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk\") pod \"network-check-target-8hwbz\" (UID: \"44861413-0803-43a6-9a94-e2e0085120c6\") " pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:20.909899 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:20.909844 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 18:59:20.909899 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:20.909866 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 18:59:20.909899 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:20.909878 2575 projected.go:194] Error preparing data for projected volume kube-api-access-kxbbk for pod openshift-network-diagnostics/network-check-target-8hwbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:20.910056 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:20.909949 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk podName:44861413-0803-43a6-9a94-e2e0085120c6 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:36.909929118 +0000 UTC m=+34.455887956 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxbbk" (UniqueName: "kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk") pod "network-check-target-8hwbz" (UID: "44861413-0803-43a6-9a94-e2e0085120c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:21.085090 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:21.084999 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:21.085273 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:21.085116 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:21.085273 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:21.085181 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:21.085399 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:21.085376 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:22.085045 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:22.085011 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:22.085540 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:22.085144 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:23.086298 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:23.086120 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:23.086726 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:23.086192 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:23.086726 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:23.086350 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:23.086726 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:23.086449 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:23.533060 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:23.532477 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:23.533060 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:23.532649 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered May 20 18:59:23.533060 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:23.532708 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret podName:1b20dd7a-c2dd-489e-bd5e-28916190af81 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:39.532690267 +0000 UTC m=+37.078649112 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret") pod "global-pull-secret-syncer-j9r94" (UID: "1b20dd7a-c2dd-489e-bd5e-28916190af81") : object "kube-system"/"original-pull-secret" not registered May 20 18:59:24.085610 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.085397 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:24.085720 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:24.085699 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:24.156439 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.156382 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" event={"ID":"21e22379-71df-4fbd-800e-e18398933ba7","Type":"ContainerStarted","Data":"3e3522d347de08dcd8aa702fc6ec619d49bb67efa593ba494459778a5928efca"} May 20 18:59:24.157652 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.157627 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" event={"ID":"f6f3ea7a-d6fb-4eb4-ace8-a22729fed55a","Type":"ContainerStarted","Data":"d47d7461c954f861d97ec6b58175e124defe8d9a59294026204ecaa200085952"} May 20 18:59:24.159795 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.159778 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 18:59:24.160082 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.160060 2575 generic.go:358] "Generic (PLEG): container finished" podID="b9b783e1-b420-4311-816e-61b4df8bb617" containerID="d4ffd1c65e7f7451693d9d48751c5a7b877ad73383f797e7a5af0f971654d5e0" exitCode=1 May 20 18:59:24.160172 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.160124 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" event={"ID":"b9b783e1-b420-4311-816e-61b4df8bb617","Type":"ContainerStarted","Data":"990c09561e8c16c85b097c3502bf2fd93357944589329b3404d134f53a3caf25"} May 20 18:59:24.160172 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.160144 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" event={"ID":"b9b783e1-b420-4311-816e-61b4df8bb617","Type":"ContainerStarted","Data":"009b7bc5b5d0066ac87b52b584b435f0fa9a83eb2161dd0d19cdd9cb10c1b509"} May 20 18:59:24.160172 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.160157 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" event={"ID":"b9b783e1-b420-4311-816e-61b4df8bb617","Type":"ContainerStarted","Data":"e74e397f4241abb5c5bb654ebab25eb2588df90980869477c9f7737f775a34b8"} May 20 18:59:24.160448 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.160172 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" event={"ID":"b9b783e1-b420-4311-816e-61b4df8bb617","Type":"ContainerDied","Data":"d4ffd1c65e7f7451693d9d48751c5a7b877ad73383f797e7a5af0f971654d5e0"} May 20 18:59:24.160448 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.160185 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" event={"ID":"b9b783e1-b420-4311-816e-61b4df8bb617","Type":"ContainerStarted","Data":"22bc08d8d40ca8c03f11e07e2c2a7b52174125e5185847b8c291594e195f4d99"} May 20 18:59:24.161600 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.161558 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d5z52" event={"ID":"f2e38623-90a4-414d-9517-087b938f83c1","Type":"ContainerStarted","Data":"ceeebea6c69fd69249256f445b03a5d3d92e633d7def3169efea8327b840a8d5"} May 20 18:59:24.163123 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.163098 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xlhks" event={"ID":"a6c9126b-be8e-4c5f-b67b-940897245170","Type":"ContainerStarted","Data":"572cdc170f36ac8ec6e54cd24b2382ca8d90df8b1015ffc20e5c8416011d7bd1"} May 20 18:59:24.164813 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.164790 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-klzxc" event={"ID":"e8bdc28d-e9ef-4ce6-9a83-1947176e82a4","Type":"ContainerStarted","Data":"f518a518a85534d453057dff1c8059756df80a1b8b23b017a243d5015df3bade"} May 20 18:59:24.169552 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.169531 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gpvmh" event={"ID":"23268351-bee9-4a36-937b-5d3eccdf2806","Type":"ContainerStarted","Data":"9928d7a6bd2467ab256e1c6db28fbb61c376a405a71c7378d8a814b929a430be"} May 20 18:59:24.171343 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.171317 2575 generic.go:358] "Generic (PLEG): container finished" podID="1c634114-53bc-40fb-abff-081af0ff74eb" containerID="8692c9fdbde2eaa9e2d5ee1f875fbec94b6d0dcc7e398ab5a62ff12f25c0de1f" exitCode=0 May 20 18:59:24.171888 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.171864 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7fszs" event={"ID":"1c634114-53bc-40fb-abff-081af0ff74eb","Type":"ContainerDied","Data":"8692c9fdbde2eaa9e2d5ee1f875fbec94b6d0dcc7e398ab5a62ff12f25c0de1f"} May 20 18:59:24.175098 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.175057 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8bcz2" podStartSLOduration=3.627313464 podStartE2EDuration="21.175047911s" podCreationTimestamp="2026-05-20 18:59:03 +0000 UTC" firstStartedPulling="2026-05-20 18:59:05.836751788 +0000 UTC m=+3.382710621" lastFinishedPulling="2026-05-20 18:59:23.384486221 +0000 UTC m=+20.930445068" observedRunningTime="2026-05-20 18:59:24.17493674 +0000 UTC m=+21.720895595" watchObservedRunningTime="2026-05-20 18:59:24.175047911 +0000 UTC m=+21.721006763" May 20 18:59:24.175605 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.175578 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-66.ec2.internal" podStartSLOduration=20.175568988 podStartE2EDuration="20.175568988s" podCreationTimestamp="2026-05-20 18:59:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 18:59:08.149428253 +0000 UTC m=+5.695387104" watchObservedRunningTime="2026-05-20 18:59:24.175568988 +0000 UTC m=+21.721527840" May 20 18:59:24.193345 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.193299 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-xlhks" podStartSLOduration=3.602292882 podStartE2EDuration="21.19328429s" podCreationTimestamp="2026-05-20 18:59:03 +0000 UTC" firstStartedPulling="2026-05-20 18:59:05.831606024 +0000 UTC m=+3.377564875" lastFinishedPulling="2026-05-20 18:59:23.422597433 +0000 UTC m=+20.968556283" observedRunningTime="2026-05-20 18:59:24.192753598 +0000 UTC m=+21.738712453" watchObservedRunningTime="2026-05-20 18:59:24.19328429 +0000 UTC m=+21.739243144" May 20 18:59:24.208229 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.208155 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-klzxc" podStartSLOduration=7.432514549 podStartE2EDuration="21.208141826s" podCreationTimestamp="2026-05-20 18:59:03 +0000 UTC" firstStartedPulling="2026-05-20 18:59:05.837614455 +0000 UTC m=+3.383573304" lastFinishedPulling="2026-05-20 18:59:19.613241736 +0000 UTC m=+17.159200581" observedRunningTime="2026-05-20 18:59:24.207836777 +0000 UTC m=+21.753795632" watchObservedRunningTime="2026-05-20 18:59:24.208141826 +0000 UTC m=+21.754100696" May 20 18:59:24.244424 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.244381 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-gpvmh" podStartSLOduration=3.697127433 podStartE2EDuration="21.244368057s" podCreationTimestamp="2026-05-20 18:59:03 +0000 UTC" firstStartedPulling="2026-05-20 18:59:05.83502274 +0000 UTC m=+3.380981573" lastFinishedPulling="2026-05-20 18:59:23.382263359 +0000 UTC m=+20.928222197" observedRunningTime="2026-05-20 18:59:24.24397636 +0000 UTC m=+21.789935216" watchObservedRunningTime="2026-05-20 18:59:24.244368057 +0000 UTC m=+21.790326911" May 20 18:59:24.258943 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.258902 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-d5z52" podStartSLOduration=3.703751225 podStartE2EDuration="21.258888059s" podCreationTimestamp="2026-05-20 18:59:03 +0000 UTC" firstStartedPulling="2026-05-20 18:59:05.827595455 +0000 UTC m=+3.373554289" lastFinishedPulling="2026-05-20 18:59:23.382732291 +0000 UTC m=+20.928691123" observedRunningTime="2026-05-20 18:59:24.258394744 +0000 UTC m=+21.804353597" watchObservedRunningTime="2026-05-20 18:59:24.258888059 +0000 UTC m=+21.804846912" May 20 18:59:24.764920 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:24.764709 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 20 18:59:25.033137 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:25.032894 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-20T18:59:24.764906435Z","UUID":"2965679d-968e-442b-b41f-ad6280c37b4a","Handler":null,"Name":"","Endpoint":""} May 20 18:59:25.035916 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:25.035894 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 20 18:59:25.036038 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:25.035924 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 20 18:59:25.085371 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:25.085339 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:25.085531 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:25.085388 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:25.085531 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:25.085469 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:25.085660 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:25.085633 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:25.175413 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:25.175346 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" event={"ID":"21e22379-71df-4fbd-800e-e18398933ba7","Type":"ContainerStarted","Data":"db905d26d17dcc60147a36895482158b4efa90ac3c8ee3037162dd7a485e3239"} May 20 18:59:25.176886 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:25.176859 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wslfb" event={"ID":"7c61a856-a85c-44a6-887d-7686c77f2eca","Type":"ContainerStarted","Data":"7ae6e912fb28e9d8d11155c83ce147ec93774c9287d0a9e38e8dfc50e86da2cd"} May 20 18:59:25.179826 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:25.179808 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 18:59:25.180253 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:25.180217 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" event={"ID":"b9b783e1-b420-4311-816e-61b4df8bb617","Type":"ContainerStarted","Data":"ef7330a4f5ea7a99e8d300307785024a2791cce9c8f6e00f5b8ecb2654d2b601"} May 20 18:59:25.193056 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:25.193019 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wslfb" podStartSLOduration=4.64121904 podStartE2EDuration="22.193006653s" podCreationTimestamp="2026-05-20 18:59:03 +0000 UTC" firstStartedPulling="2026-05-20 18:59:05.830481103 +0000 UTC m=+3.376439942" lastFinishedPulling="2026-05-20 18:59:23.382268718 +0000 UTC m=+20.928227555" observedRunningTime="2026-05-20 18:59:25.192969724 +0000 UTC m=+22.738928578" watchObservedRunningTime="2026-05-20 18:59:25.193006653 +0000 UTC m=+22.738965506" May 20 18:59:26.085056 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:26.085021 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:26.085229 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:26.085133 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:26.184975 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:26.184893 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" event={"ID":"21e22379-71df-4fbd-800e-e18398933ba7","Type":"ContainerStarted","Data":"c2cc1181c5217d6ec3dcd2546cffbfd8037c1b9f76624c0459602969113190a9"} May 20 18:59:26.202315 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:26.202260 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9jpb" podStartSLOduration=3.133633396 podStartE2EDuration="23.202245261s" podCreationTimestamp="2026-05-20 18:59:03 +0000 UTC" firstStartedPulling="2026-05-20 18:59:05.837710002 +0000 UTC m=+3.383668840" lastFinishedPulling="2026-05-20 18:59:25.906321873 +0000 UTC m=+23.452280705" observedRunningTime="2026-05-20 18:59:26.201575581 +0000 UTC m=+23.747534435" watchObservedRunningTime="2026-05-20 18:59:26.202245261 +0000 UTC m=+23.748204114" May 20 18:59:26.587420 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:26.587383 2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-klzxc" May 20 18:59:26.587980 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:26.587956 2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-klzxc" May 20 18:59:27.085594 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:27.085561 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:27.085792 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:27.085562 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:27.085792 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:27.085681 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:27.085792 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:27.085753 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:27.189695 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:27.189662 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 18:59:27.190329 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:27.190070 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" event={"ID":"b9b783e1-b420-4311-816e-61b4df8bb617","Type":"ContainerStarted","Data":"551fff4a8728fdb2c6d77f641e2925c50ecc2c6035fa3bff1adf0fa0fb14bd57"} May 20 18:59:27.190505 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:27.190483 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-klzxc" May 20 18:59:27.191061 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:27.191044 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-klzxc" May 20 18:59:28.085264 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:28.085232 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:28.085412 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:28.085338 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:29.088226 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:29.088052 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:29.088652 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:29.088053 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:29.088652 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:29.088301 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:29.088652 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:29.088370 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:29.195658 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:29.195628 2575 generic.go:358] "Generic (PLEG): container finished" podID="1c634114-53bc-40fb-abff-081af0ff74eb" containerID="d51ec25d74a294d51258f44183b308c6996efb3a4a8d19a84545a67db2d37a01" exitCode=0 May 20 18:59:29.195815 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:29.195720 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7fszs" event={"ID":"1c634114-53bc-40fb-abff-081af0ff74eb","Type":"ContainerDied","Data":"d51ec25d74a294d51258f44183b308c6996efb3a4a8d19a84545a67db2d37a01"} May 20 18:59:29.198844 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:29.198827 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 18:59:29.199174 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:29.199149 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" event={"ID":"b9b783e1-b420-4311-816e-61b4df8bb617","Type":"ContainerStarted","Data":"147ac59df0f274aabb58f0d60d0294981ad6115d752a7d4c9a57130b84dffdc4"} May 20 18:59:29.199681 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:29.199663 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:29.199757 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:29.199693 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:29.199798 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:29.199778 2575 scope.go:117] "RemoveContainer" containerID="d4ffd1c65e7f7451693d9d48751c5a7b877ad73383f797e7a5af0f971654d5e0" May 20 18:59:29.216090 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:29.216069 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:30.085360 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.085332 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:30.085531 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:30.085426 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:30.203562 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.203536 2575 generic.go:358] "Generic (PLEG): container finished" podID="1c634114-53bc-40fb-abff-081af0ff74eb" containerID="3028195aa1390ab467abb2194c1c1881ae5c96d7cc814d85d8bd2a4f40cba3ed" exitCode=0 May 20 18:59:30.204018 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.203618 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7fszs" event={"ID":"1c634114-53bc-40fb-abff-081af0ff74eb","Type":"ContainerDied","Data":"3028195aa1390ab467abb2194c1c1881ae5c96d7cc814d85d8bd2a4f40cba3ed"} May 20 18:59:30.207410 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.207394 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 18:59:30.207706 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.207684 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" event={"ID":"b9b783e1-b420-4311-816e-61b4df8bb617","Type":"ContainerStarted","Data":"c6bdf44cdc55c05d9b12ed1890d80a1b60bb95ee3c999fbbf1e809cd7625d4c9"} May 20 18:59:30.207957 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.207937 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:30.222448 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.222427 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 18:59:30.268735 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.268685 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" podStartSLOduration=9.69277513 podStartE2EDuration="27.26867071s" podCreationTimestamp="2026-05-20 18:59:03 +0000 UTC" firstStartedPulling="2026-05-20 18:59:05.828543475 +0000 UTC m=+3.374502312" lastFinishedPulling="2026-05-20 18:59:23.404439042 +0000 UTC m=+20.950397892" observedRunningTime="2026-05-20 18:59:30.268423847 +0000 UTC m=+27.814382726" watchObservedRunningTime="2026-05-20 18:59:30.26867071 +0000 UTC m=+27.814629563" May 20 18:59:30.397110 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.397079 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j9r94"] May 20 18:59:30.397300 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.397199 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:30.397369 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:30.397305 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:30.399835 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.399809 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8hwbz"] May 20 18:59:30.399971 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.399911 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:30.400033 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:30.399991 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:30.408496 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.408468 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-84tr8"] May 20 18:59:30.408602 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:30.408589 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:30.408713 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:30.408694 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:31.211687 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:31.211655 2575 generic.go:358] "Generic (PLEG): container finished" podID="1c634114-53bc-40fb-abff-081af0ff74eb" containerID="cfcf15e3cab22ab7155d553738b122be8ecf195fecd59de3f90e031c04dd0837" exitCode=0 May 20 18:59:31.212150 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:31.211738 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7fszs" event={"ID":"1c634114-53bc-40fb-abff-081af0ff74eb","Type":"ContainerDied","Data":"cfcf15e3cab22ab7155d553738b122be8ecf195fecd59de3f90e031c04dd0837"} May 20 18:59:32.085224 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:32.085174 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:32.085224 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:32.085197 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:32.085224 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:32.085174 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:32.085515 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:32.085319 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:32.085515 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:32.085400 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:32.085515 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:32.085464 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:34.085324 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:34.085270 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:34.085955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:34.085269 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:34.085955 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:34.085422 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:34.085955 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:34.085273 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:34.085955 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:34.085479 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:34.085955 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:34.085558 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:36.084872 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:36.084841 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:36.085396 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:36.084841 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:36.085396 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:36.084964 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j9r94" podUID="1b20dd7a-c2dd-489e-bd5e-28916190af81" May 20 18:59:36.085396 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:36.084841 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:36.085396 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:36.085070 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 18:59:36.085396 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:36.085177 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hwbz" podUID="44861413-0803-43a6-9a94-e2e0085120c6" May 20 18:59:36.736994 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:36.736934 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:36.737191 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:36.737080 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:36.737191 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:36.737163 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs podName:8c283ba7-2436-4f29-b86c-97340c785bee nodeName:}" failed. No retries permitted until 2026-05-20 19:00:08.737141778 +0000 UTC m=+66.283100626 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs") pod "network-metrics-daemon-84tr8" (UID: "8c283ba7-2436-4f29-b86c-97340c785bee") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 18:59:36.938451 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:36.938413 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kxbbk\" (UniqueName: \"kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk\") pod \"network-check-target-8hwbz\" (UID: \"44861413-0803-43a6-9a94-e2e0085120c6\") " pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:36.938628 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:36.938541 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 18:59:36.938628 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:36.938554 2575 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 18:59:36.938628 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:36.938563 2575 projected.go:194] Error preparing data for projected volume kube-api-access-kxbbk for pod openshift-network-diagnostics/network-check-target-8hwbz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:36.938628 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:36.938611 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk podName:44861413-0803-43a6-9a94-e2e0085120c6 nodeName:}" failed. No retries permitted until 2026-05-20 19:00:08.938597183 +0000 UTC m=+66.484556015 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kxbbk" (UniqueName: "kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk") pod "network-check-target-8hwbz" (UID: "44861413-0803-43a6-9a94-e2e0085120c6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 18:59:37.198575 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.198413 2575 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-66.ec2.internal" event="NodeReady" May 20 18:59:37.198985 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.198669 2575 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 20 18:59:37.241567 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.241485 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-95f7s"] May 20 18:59:37.254281 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.254251 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lhst4"] May 20 18:59:37.254453 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.254402 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-95f7s" May 20 18:59:37.256924 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.256900 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ms74x\"" May 20 18:59:37.257088 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.257002 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 20 18:59:37.257191 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.257173 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 20 18:59:37.266689 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.266666 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-95f7s"] May 20 18:59:37.266689 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.266689 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lhst4"] May 20 18:59:37.266824 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.266775 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 18:59:37.269288 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.269269 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 20 18:59:37.269393 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.269288 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 20 18:59:37.269393 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.269307 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hwgwl\"" May 20 18:59:37.269502 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.269314 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 20 18:59:37.441963 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.441925 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:37.441963 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.441965 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d601dbc0-665a-4564-833e-bef2c554536c-tmp-dir\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:37.442190 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.441993 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 18:59:37.442190 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.442013 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58567\" (UniqueName: \"kubernetes.io/projected/d601dbc0-665a-4564-833e-bef2c554536c-kube-api-access-58567\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:37.442190 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.442031 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d601dbc0-665a-4564-833e-bef2c554536c-config-volume\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:37.442190 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.442137 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbzdx\" (UniqueName: \"kubernetes.io/projected/b4c6b123-27fc-45c3-a22f-6cf99088f182-kube-api-access-kbzdx\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 18:59:37.542874 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.542845 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kbzdx\" (UniqueName: \"kubernetes.io/projected/b4c6b123-27fc-45c3-a22f-6cf99088f182-kube-api-access-kbzdx\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 18:59:37.543035 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.542885 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:37.543035 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.542902 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d601dbc0-665a-4564-833e-bef2c554536c-tmp-dir\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:37.543035 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.542920 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 18:59:37.543035 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.542938 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-58567\" (UniqueName: \"kubernetes.io/projected/d601dbc0-665a-4564-833e-bef2c554536c-kube-api-access-58567\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:37.543035 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.542955 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d601dbc0-665a-4564-833e-bef2c554536c-config-volume\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:37.543285 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:37.543040 2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 18:59:37.543285 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:37.543092 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 18:59:37.543285 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:37.543119 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert podName:b4c6b123-27fc-45c3-a22f-6cf99088f182 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:38.043098487 +0000 UTC m=+35.589057327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert") pod "ingress-canary-lhst4" (UID: "b4c6b123-27fc-45c3-a22f-6cf99088f182") : secret "canary-serving-cert" not found May 20 18:59:37.543285 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:37.543141 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls podName:d601dbc0-665a-4564-833e-bef2c554536c nodeName:}" failed. No retries permitted until 2026-05-20 18:59:38.043127003 +0000 UTC m=+35.589085838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls") pod "dns-default-95f7s" (UID: "d601dbc0-665a-4564-833e-bef2c554536c") : secret "dns-default-metrics-tls" not found May 20 18:59:37.543285 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.543242 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d601dbc0-665a-4564-833e-bef2c554536c-tmp-dir\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:37.543534 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.543517 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d601dbc0-665a-4564-833e-bef2c554536c-config-volume\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:37.553038 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.553016 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58567\" (UniqueName: \"kubernetes.io/projected/d601dbc0-665a-4564-833e-bef2c554536c-kube-api-access-58567\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:37.553250 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:37.553228 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbzdx\" (UniqueName: \"kubernetes.io/projected/b4c6b123-27fc-45c3-a22f-6cf99088f182-kube-api-access-kbzdx\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 18:59:38.045829 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:38.045791 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:38.045829 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:38.045833 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 18:59:38.046068 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:38.045944 2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 18:59:38.046068 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:38.045949 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 18:59:38.046068 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:38.045995 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert podName:b4c6b123-27fc-45c3-a22f-6cf99088f182 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:39.045980787 +0000 UTC m=+36.591939619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert") pod "ingress-canary-lhst4" (UID: "b4c6b123-27fc-45c3-a22f-6cf99088f182") : secret "canary-serving-cert" not found May 20 18:59:38.046068 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:38.046007 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls podName:d601dbc0-665a-4564-833e-bef2c554536c nodeName:}" failed. No retries permitted until 2026-05-20 18:59:39.046001056 +0000 UTC m=+36.591959888 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls") pod "dns-default-95f7s" (UID: "d601dbc0-665a-4564-833e-bef2c554536c") : secret "dns-default-metrics-tls" not found May 20 18:59:38.085601 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:38.085566 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 18:59:38.085779 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:38.085679 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:38.085779 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:38.085698 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 18:59:38.088294 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:38.088274 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 20 18:59:38.088294 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:38.088287 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b6xjs\"" May 20 18:59:38.088467 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:38.088428 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 20 18:59:38.088467 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:38.088453 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 20 18:59:38.088572 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:38.088527 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 20 18:59:38.088781 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:38.088763 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pfj4b\"" May 20 18:59:38.229317 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:38.229284 2575 generic.go:358] "Generic (PLEG): container finished" podID="1c634114-53bc-40fb-abff-081af0ff74eb" containerID="2eafe685b9f1e9c5a99b3fffab19e5565b837cdccb14253ca1c9cdc3d0984e7c" exitCode=0 May 20 18:59:38.229317 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:38.229323 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7fszs" event={"ID":"1c634114-53bc-40fb-abff-081af0ff74eb","Type":"ContainerDied","Data":"2eafe685b9f1e9c5a99b3fffab19e5565b837cdccb14253ca1c9cdc3d0984e7c"} May 20 18:59:39.053133 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.053093 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:39.053133 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.053135 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 18:59:39.053362 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:39.053253 2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 18:59:39.053362 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:39.053255 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 18:59:39.053362 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:39.053304 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert podName:b4c6b123-27fc-45c3-a22f-6cf99088f182 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:41.053290702 +0000 UTC m=+38.599249535 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert") pod "ingress-canary-lhst4" (UID: "b4c6b123-27fc-45c3-a22f-6cf99088f182") : secret "canary-serving-cert" not found May 20 18:59:39.053362 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:39.053317 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls podName:d601dbc0-665a-4564-833e-bef2c554536c nodeName:}" failed. No retries permitted until 2026-05-20 18:59:41.053311581 +0000 UTC m=+38.599270413 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls") pod "dns-default-95f7s" (UID: "d601dbc0-665a-4564-833e-bef2c554536c") : secret "dns-default-metrics-tls" not found May 20 18:59:39.233495 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.233462 2575 generic.go:358] "Generic (PLEG): container finished" podID="1c634114-53bc-40fb-abff-081af0ff74eb" containerID="059f82b509e26c14fc1c1cb1d3c9fa836f946be5c2dc3e18875f03cbdb17701c" exitCode=0 May 20 18:59:39.233864 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.233523 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7fszs" event={"ID":"1c634114-53bc-40fb-abff-081af0ff74eb","Type":"ContainerDied","Data":"059f82b509e26c14fc1c1cb1d3c9fa836f946be5c2dc3e18875f03cbdb17701c"} May 20 18:59:39.556492 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.556454 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:39.559646 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.559506 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1b20dd7a-c2dd-489e-bd5e-28916190af81-original-pull-secret\") pod \"global-pull-secret-syncer-j9r94\" (UID: \"1b20dd7a-c2dd-489e-bd5e-28916190af81\") " pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:39.592401 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.592317 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8"] May 20 18:59:39.600657 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.600623 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j9r94" May 20 18:59:39.607914 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.606995 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln"] May 20 18:59:39.607914 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.607167 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8" May 20 18:59:39.609632 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.609610 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-t4rhp\"" May 20 18:59:39.610050 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.610033 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" May 20 18:59:39.610153 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.610135 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" May 20 18:59:39.610234 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.610196 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" May 20 18:59:39.610633 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.610613 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" May 20 18:59:39.625340 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.625315 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8"] May 20 18:59:39.625340 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.625342 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln"] May 20 18:59:39.625503 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.625444 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.627962 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.627938 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" May 20 18:59:39.628097 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.628028 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" May 20 18:59:39.628159 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.628137 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" May 20 18:59:39.628253 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.628237 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" May 20 18:59:39.758630 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.758600 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/3ff0165c-767f-431c-8f22-de468176c86e-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.758798 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.758638 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ff0165c-767f-431c-8f22-de468176c86e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.758798 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.758684 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3ff0165c-767f-431c-8f22-de468176c86e-ca\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.758798 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.758711 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3ff0165c-767f-431c-8f22-de468176c86e-hub\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.758798 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.758729 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3ff0165c-767f-431c-8f22-de468176c86e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.758953 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.758818 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7p6t\" (UniqueName: \"kubernetes.io/projected/3ff0165c-767f-431c-8f22-de468176c86e-kube-api-access-r7p6t\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.758953 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.758859 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7km2\" (UniqueName: \"kubernetes.io/projected/e41815e6-705b-4f8b-b02d-87b2bb87be6c-kube-api-access-z7km2\") pod \"managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8\" (UID: \"e41815e6-705b-4f8b-b02d-87b2bb87be6c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8" May 20 18:59:39.758953 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.758896 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e41815e6-705b-4f8b-b02d-87b2bb87be6c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8\" (UID: \"e41815e6-705b-4f8b-b02d-87b2bb87be6c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8" May 20 18:59:39.792470 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.790995 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j9r94"] May 20 18:59:39.796294 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:39.796265 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b20dd7a_c2dd_489e_bd5e_28916190af81.slice/crio-9e703c933846184f9cdbf344d309f957896f74c0298f6450628c9cd7bae77610 WatchSource:0}: Error finding container 9e703c933846184f9cdbf344d309f957896f74c0298f6450628c9cd7bae77610: Status 404 returned error can't find the container with id 9e703c933846184f9cdbf344d309f957896f74c0298f6450628c9cd7bae77610 May 20 18:59:39.859829 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.859734 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ff0165c-767f-431c-8f22-de468176c86e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.859829 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.859803 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3ff0165c-767f-431c-8f22-de468176c86e-ca\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.860046 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.859848 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3ff0165c-767f-431c-8f22-de468176c86e-hub\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.860046 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.859870 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3ff0165c-767f-431c-8f22-de468176c86e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.860046 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.859921 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r7p6t\" (UniqueName: \"kubernetes.io/projected/3ff0165c-767f-431c-8f22-de468176c86e-kube-api-access-r7p6t\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.860046 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.859944 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z7km2\" (UniqueName: \"kubernetes.io/projected/e41815e6-705b-4f8b-b02d-87b2bb87be6c-kube-api-access-z7km2\") pod \"managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8\" (UID: \"e41815e6-705b-4f8b-b02d-87b2bb87be6c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8" May 20 18:59:39.860046 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.859983 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e41815e6-705b-4f8b-b02d-87b2bb87be6c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8\" (UID: \"e41815e6-705b-4f8b-b02d-87b2bb87be6c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8" May 20 18:59:39.860046 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.860013 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/3ff0165c-767f-431c-8f22-de468176c86e-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.860793 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.860769 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/3ff0165c-767f-431c-8f22-de468176c86e-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.862816 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.862786 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3ff0165c-767f-431c-8f22-de468176c86e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.863109 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.863091 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3ff0165c-767f-431c-8f22-de468176c86e-hub\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.863191 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.863172 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3ff0165c-767f-431c-8f22-de468176c86e-ca\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.863253 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.863238 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ff0165c-767f-431c-8f22-de468176c86e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.863289 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.863238 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e41815e6-705b-4f8b-b02d-87b2bb87be6c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8\" (UID: \"e41815e6-705b-4f8b-b02d-87b2bb87be6c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8" May 20 18:59:39.868418 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.868369 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7p6t\" (UniqueName: \"kubernetes.io/projected/3ff0165c-767f-431c-8f22-de468176c86e-kube-api-access-r7p6t\") pod \"cluster-proxy-proxy-agent-77cf746c47-tzpln\" (UID: \"3ff0165c-767f-431c-8f22-de468176c86e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:39.868516 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.868461 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7km2\" (UniqueName: \"kubernetes.io/projected/e41815e6-705b-4f8b-b02d-87b2bb87be6c-kube-api-access-z7km2\") pod \"managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8\" (UID: \"e41815e6-705b-4f8b-b02d-87b2bb87be6c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8" May 20 18:59:39.932839 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.932800 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8" May 20 18:59:39.938673 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:39.938646 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 18:59:40.097346 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:40.097313 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln"] May 20 18:59:40.101279 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:40.101248 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ff0165c_767f_431c_8f22_de468176c86e.slice/crio-720fba8cf13127d3c3e2219d5c3a1fb3b96e338784229ab6d2c78dc61fa5a1ed WatchSource:0}: Error finding container 720fba8cf13127d3c3e2219d5c3a1fb3b96e338784229ab6d2c78dc61fa5a1ed: Status 404 returned error can't find the container with id 720fba8cf13127d3c3e2219d5c3a1fb3b96e338784229ab6d2c78dc61fa5a1ed May 20 18:59:40.107751 ip-10-0-138-66 kubenswrapper[2575]: W0520 18:59:40.107725 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode41815e6_705b_4f8b_b02d_87b2bb87be6c.slice/crio-066fe727bb90f09bae4e363fd224b80169489308cdc71320cb6d2e0cd2f14749 WatchSource:0}: Error finding container 066fe727bb90f09bae4e363fd224b80169489308cdc71320cb6d2e0cd2f14749: Status 404 returned error can't find the container with id 066fe727bb90f09bae4e363fd224b80169489308cdc71320cb6d2e0cd2f14749 May 20 18:59:40.111120 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:40.111097 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8"] May 20 18:59:40.237371 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:40.237331 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" event={"ID":"3ff0165c-767f-431c-8f22-de468176c86e","Type":"ContainerStarted","Data":"720fba8cf13127d3c3e2219d5c3a1fb3b96e338784229ab6d2c78dc61fa5a1ed"} May 20 18:59:40.238528 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:40.238505 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8" event={"ID":"e41815e6-705b-4f8b-b02d-87b2bb87be6c","Type":"ContainerStarted","Data":"066fe727bb90f09bae4e363fd224b80169489308cdc71320cb6d2e0cd2f14749"} May 20 18:59:40.241777 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:40.241750 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7fszs" event={"ID":"1c634114-53bc-40fb-abff-081af0ff74eb","Type":"ContainerStarted","Data":"94be915d1e16ffe6c9000e8fc8ca22815345dc4d533319331751e1f89096155b"} May 20 18:59:40.243101 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:40.243079 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j9r94" event={"ID":"1b20dd7a-c2dd-489e-bd5e-28916190af81","Type":"ContainerStarted","Data":"9e703c933846184f9cdbf344d309f957896f74c0298f6450628c9cd7bae77610"} May 20 18:59:41.069463 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:41.069407 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 18:59:41.069657 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:41.069527 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:41.069657 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:41.069576 2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 18:59:41.069657 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:41.069634 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 18:59:41.069807 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:41.069661 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert podName:b4c6b123-27fc-45c3-a22f-6cf99088f182 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:45.069639625 +0000 UTC m=+42.615598470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert") pod "ingress-canary-lhst4" (UID: "b4c6b123-27fc-45c3-a22f-6cf99088f182") : secret "canary-serving-cert" not found May 20 18:59:41.069807 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:41.069679 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls podName:d601dbc0-665a-4564-833e-bef2c554536c nodeName:}" failed. No retries permitted until 2026-05-20 18:59:45.069670845 +0000 UTC m=+42.615629688 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls") pod "dns-default-95f7s" (UID: "d601dbc0-665a-4564-833e-bef2c554536c") : secret "dns-default-metrics-tls" not found May 20 18:59:43.119429 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:43.118496 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7fszs" podStartSLOduration=8.675907059 podStartE2EDuration="40.118476568s" podCreationTimestamp="2026-05-20 18:59:03 +0000 UTC" firstStartedPulling="2026-05-20 18:59:05.827558033 +0000 UTC m=+3.373516873" lastFinishedPulling="2026-05-20 18:59:37.27012755 +0000 UTC m=+34.816086382" observedRunningTime="2026-05-20 18:59:40.267904512 +0000 UTC m=+37.813863362" watchObservedRunningTime="2026-05-20 18:59:43.118476568 +0000 UTC m=+40.664435422" May 20 18:59:45.099007 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:45.098979 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:45.099492 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:45.099019 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 18:59:45.099492 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:45.099147 2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 18:59:45.099492 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:45.099193 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert podName:b4c6b123-27fc-45c3-a22f-6cf99088f182 nodeName:}" failed. No retries permitted until 2026-05-20 18:59:53.099180227 +0000 UTC m=+50.645139059 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert") pod "ingress-canary-lhst4" (UID: "b4c6b123-27fc-45c3-a22f-6cf99088f182") : secret "canary-serving-cert" not found May 20 18:59:45.099492 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:45.099147 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 18:59:45.099765 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:45.099508 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls podName:d601dbc0-665a-4564-833e-bef2c554536c nodeName:}" failed. No retries permitted until 2026-05-20 18:59:53.099494333 +0000 UTC m=+50.645453165 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls") pod "dns-default-95f7s" (UID: "d601dbc0-665a-4564-833e-bef2c554536c") : secret "dns-default-metrics-tls" not found May 20 18:59:45.255825 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:45.255715 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j9r94" event={"ID":"1b20dd7a-c2dd-489e-bd5e-28916190af81","Type":"ContainerStarted","Data":"026cb904faf38bc82295733a5ff33770e47ff1d69ea433bbc394bece048aae70"} May 20 18:59:45.257175 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:45.257147 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8" event={"ID":"e41815e6-705b-4f8b-b02d-87b2bb87be6c","Type":"ContainerStarted","Data":"066d77b2870f5f344ac1adcebaef0df4f3f1b7d85147a59b57558972c61f52d7"} May 20 18:59:45.272702 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:45.272635 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j9r94" podStartSLOduration=33.074746412 podStartE2EDuration="38.272613956s" podCreationTimestamp="2026-05-20 18:59:07 +0000 UTC" firstStartedPulling="2026-05-20 18:59:39.79829789 +0000 UTC m=+37.344256722" lastFinishedPulling="2026-05-20 18:59:44.99616542 +0000 UTC m=+42.542124266" observedRunningTime="2026-05-20 18:59:45.271915696 +0000 UTC m=+42.817874548" watchObservedRunningTime="2026-05-20 18:59:45.272613956 +0000 UTC m=+42.818572808" May 20 18:59:45.286274 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:45.286197 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8" podStartSLOduration=1.414122068 podStartE2EDuration="6.286175638s" podCreationTimestamp="2026-05-20 18:59:39 +0000 UTC" firstStartedPulling="2026-05-20 18:59:40.109882457 +0000 UTC m=+37.655841291" lastFinishedPulling="2026-05-20 18:59:44.981936019 +0000 UTC m=+42.527894861" observedRunningTime="2026-05-20 18:59:45.285557257 +0000 UTC m=+42.831516111" watchObservedRunningTime="2026-05-20 18:59:45.286175638 +0000 UTC m=+42.832134494" May 20 18:59:47.264334 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:47.264245 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" event={"ID":"3ff0165c-767f-431c-8f22-de468176c86e","Type":"ContainerStarted","Data":"a8a1d99af55c589ac6ec3a9edd5b072bb350bff6b87e253c6bd8b0d085a38a64"} May 20 18:59:53.163279 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:53.163240 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 18:59:53.163279 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:53.163286 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 18:59:53.163655 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:53.163386 2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 18:59:53.163655 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:53.163388 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 18:59:53.163655 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:53.163436 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert podName:b4c6b123-27fc-45c3-a22f-6cf99088f182 nodeName:}" failed. No retries permitted until 2026-05-20 19:00:09.163423328 +0000 UTC m=+66.709382160 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert") pod "ingress-canary-lhst4" (UID: "b4c6b123-27fc-45c3-a22f-6cf99088f182") : secret "canary-serving-cert" not found May 20 18:59:53.163655 ip-10-0-138-66 kubenswrapper[2575]: E0520 18:59:53.163447 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls podName:d601dbc0-665a-4564-833e-bef2c554536c nodeName:}" failed. No retries permitted until 2026-05-20 19:00:09.163442334 +0000 UTC m=+66.709401166 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls") pod "dns-default-95f7s" (UID: "d601dbc0-665a-4564-833e-bef2c554536c") : secret "dns-default-metrics-tls" not found May 20 18:59:53.277600 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:53.277560 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" event={"ID":"3ff0165c-767f-431c-8f22-de468176c86e","Type":"ContainerStarted","Data":"0f4476d49c703ffad55b29fa2776333c58c5b142589665f7be2b2c784475010d"} May 20 18:59:53.277600 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:53.277604 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" event={"ID":"3ff0165c-767f-431c-8f22-de468176c86e","Type":"ContainerStarted","Data":"b826b88950c4383fafe64abd846808349319035cd835db20fb7cf6756ad78478"} May 20 18:59:53.296314 ip-10-0-138-66 kubenswrapper[2575]: I0520 18:59:53.296263 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" podStartSLOduration=2.179313473 podStartE2EDuration="14.296250456s" podCreationTimestamp="2026-05-20 18:59:39 +0000 UTC" firstStartedPulling="2026-05-20 18:59:40.102965613 +0000 UTC m=+37.648924449" lastFinishedPulling="2026-05-20 18:59:52.219902598 +0000 UTC m=+49.765861432" observedRunningTime="2026-05-20 18:59:53.294791742 +0000 UTC m=+50.840750596" watchObservedRunningTime="2026-05-20 18:59:53.296250456 +0000 UTC m=+50.842209309" May 20 19:00:02.223920 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:02.223891 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-szh8j" May 20 19:00:08.775809 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:08.775760 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 19:00:08.778934 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:08.778914 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 20 19:00:08.786291 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:00:08.786269 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 20 19:00:08.786358 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:00:08.786338 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs podName:8c283ba7-2436-4f29-b86c-97340c785bee nodeName:}" failed. No retries permitted until 2026-05-20 19:01:12.786322773 +0000 UTC m=+130.332281605 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs") pod "network-metrics-daemon-84tr8" (UID: "8c283ba7-2436-4f29-b86c-97340c785bee") : secret "metrics-daemon-secret" not found May 20 19:00:08.977903 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:08.977861 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kxbbk\" (UniqueName: \"kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk\") pod \"network-check-target-8hwbz\" (UID: \"44861413-0803-43a6-9a94-e2e0085120c6\") " pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 19:00:08.981027 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:08.980996 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 20 19:00:08.990477 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:08.990451 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 20 19:00:09.001187 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:09.001163 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxbbk\" (UniqueName: \"kubernetes.io/projected/44861413-0803-43a6-9a94-e2e0085120c6-kube-api-access-kxbbk\") pod \"network-check-target-8hwbz\" (UID: \"44861413-0803-43a6-9a94-e2e0085120c6\") " pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 19:00:09.179552 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:09.179510 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 19:00:09.179552 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:09.179557 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 19:00:09.179808 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:00:09.179675 2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 19:00:09.179808 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:00:09.179683 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 19:00:09.179808 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:00:09.179735 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert podName:b4c6b123-27fc-45c3-a22f-6cf99088f182 nodeName:}" failed. No retries permitted until 2026-05-20 19:00:41.179721764 +0000 UTC m=+98.725680596 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert") pod "ingress-canary-lhst4" (UID: "b4c6b123-27fc-45c3-a22f-6cf99088f182") : secret "canary-serving-cert" not found May 20 19:00:09.179808 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:00:09.179747 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls podName:d601dbc0-665a-4564-833e-bef2c554536c nodeName:}" failed. No retries permitted until 2026-05-20 19:00:41.179742053 +0000 UTC m=+98.725700885 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls") pod "dns-default-95f7s" (UID: "d601dbc0-665a-4564-833e-bef2c554536c") : secret "dns-default-metrics-tls" not found May 20 19:00:09.298163 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:09.298134 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pfj4b\"" May 20 19:00:09.306454 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:09.306418 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 19:00:09.426978 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:09.426944 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8hwbz"] May 20 19:00:09.430391 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:00:09.430321 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44861413_0803_43a6_9a94_e2e0085120c6.slice/crio-1e0c57e1f331d17f0dd61dd1c0b01c6a0179d27c5ecf015485704611bca943a5 WatchSource:0}: Error finding container 1e0c57e1f331d17f0dd61dd1c0b01c6a0179d27c5ecf015485704611bca943a5: Status 404 returned error can't find the container with id 1e0c57e1f331d17f0dd61dd1c0b01c6a0179d27c5ecf015485704611bca943a5 May 20 19:00:10.314539 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:10.314502 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8hwbz" event={"ID":"44861413-0803-43a6-9a94-e2e0085120c6","Type":"ContainerStarted","Data":"1e0c57e1f331d17f0dd61dd1c0b01c6a0179d27c5ecf015485704611bca943a5"} May 20 19:00:12.319758 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:12.319721 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8hwbz" event={"ID":"44861413-0803-43a6-9a94-e2e0085120c6","Type":"ContainerStarted","Data":"9d520bbba451f794e9b5853975de9fdfeef2e3b04cece4b7175541475bc0226a"} May 20 19:00:12.320186 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:12.319849 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 19:00:12.342113 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:12.342005 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8hwbz" podStartSLOduration=66.685065837 podStartE2EDuration="1m9.341987263s" podCreationTimestamp="2026-05-20 18:59:03 +0000 UTC" firstStartedPulling="2026-05-20 19:00:09.432644224 +0000 UTC m=+66.978603057" lastFinishedPulling="2026-05-20 19:00:12.089565647 +0000 UTC m=+69.635524483" observedRunningTime="2026-05-20 19:00:12.341467058 +0000 UTC m=+69.887425912" watchObservedRunningTime="2026-05-20 19:00:12.341987263 +0000 UTC m=+69.887946118" May 20 19:00:41.219503 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:41.219449 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 19:00:41.219503 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:41.219504 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 19:00:41.219924 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:00:41.219599 2575 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 19:00:41.219924 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:00:41.219603 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 19:00:41.219924 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:00:41.219652 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert podName:b4c6b123-27fc-45c3-a22f-6cf99088f182 nodeName:}" failed. No retries permitted until 2026-05-20 19:01:45.219638681 +0000 UTC m=+162.765597514 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert") pod "ingress-canary-lhst4" (UID: "b4c6b123-27fc-45c3-a22f-6cf99088f182") : secret "canary-serving-cert" not found May 20 19:00:41.219924 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:00:41.219665 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls podName:d601dbc0-665a-4564-833e-bef2c554536c nodeName:}" failed. No retries permitted until 2026-05-20 19:01:45.219658579 +0000 UTC m=+162.765617411 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls") pod "dns-default-95f7s" (UID: "d601dbc0-665a-4564-833e-bef2c554536c") : secret "dns-default-metrics-tls" not found May 20 19:00:43.324288 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:00:43.324254 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8hwbz" May 20 19:01:12.835866 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:12.835821 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 19:01:12.836328 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:12.835976 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 20 19:01:12.836328 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:12.836061 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs podName:8c283ba7-2436-4f29-b86c-97340c785bee nodeName:}" failed. No retries permitted until 2026-05-20 19:03:14.83604205 +0000 UTC m=+252.382000883 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs") pod "network-metrics-daemon-84tr8" (UID: "8c283ba7-2436-4f29-b86c-97340c785bee") : secret "metrics-daemon-secret" not found May 20 19:01:18.839120 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:18.839090 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gpvmh_23268351-bee9-4a36-937b-5d3eccdf2806/dns-node-resolver/0.log" May 20 19:01:19.637965 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:19.637938 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d5z52_f2e38623-90a4-414d-9517-087b938f83c1/node-ca/0.log" May 20 19:01:40.265689 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:40.265597 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-95f7s" podUID="d601dbc0-665a-4564-833e-bef2c554536c" May 20 19:01:40.275776 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:40.275737 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-lhst4" podUID="b4c6b123-27fc-45c3-a22f-6cf99088f182" May 20 19:01:40.528295 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:40.528192 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-95f7s" May 20 19:01:41.105880 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:41.105830 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-84tr8" podUID="8c283ba7-2436-4f29-b86c-97340c785bee" May 20 19:01:44.423406 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.423363 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5kdhc"] May 20 19:01:44.425530 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.425509 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:44.435657 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:44.435626 2575 reflector.go:200] "Failed to watch" err="failed to list *v1.Secret: secrets \"insights-runtime-extractor-tls\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret" May 20 19:01:44.439359 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:44.439334 2575 reflector.go:200] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" May 20 19:01:44.441371 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:44.441349 2575 reflector.go:200] "Failed to watch" err="failed to list *v1.Secret: secrets \"insights-runtime-extractor-sa-dockercfg-d28f9\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d28f9\"" type="*v1.Secret" May 20 19:01:44.441443 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.441423 2575 status_manager.go:895] "Failed to get status for pod" podUID="e31f8582-cd37-4c6f-a313-cd02ca18d675" pod="openshift-insights/insights-runtime-extractor-5kdhc" err="pods \"insights-runtime-extractor-5kdhc\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" May 20 19:01:44.441566 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:44.441551 2575 reflector.go:200] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" May 20 19:01:44.462026 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.461998 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e31f8582-cd37-4c6f-a313-cd02ca18d675-crio-socket\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:44.462172 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.462053 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e31f8582-cd37-4c6f-a313-cd02ca18d675-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:44.462172 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.462074 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e31f8582-cd37-4c6f-a313-cd02ca18d675-data-volume\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:44.462172 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.462093 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e31f8582-cd37-4c6f-a313-cd02ca18d675-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:44.462172 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.462161 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24fkf\" (UniqueName: \"kubernetes.io/projected/e31f8582-cd37-4c6f-a313-cd02ca18d675-kube-api-access-24fkf\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:44.498838 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:44.498795 2575 reflector.go:200] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-rbac-proxy\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" type="*v1.ConfigMap" May 20 19:01:44.500672 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.500640 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66dc6fb849-rxgv6"] May 20 19:01:44.502352 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.502337 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5kdhc"] May 20 19:01:44.502442 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.502432 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.510146 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.510125 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" May 20 19:01:44.548680 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.548654 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" May 20 19:01:44.550752 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.550734 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" May 20 19:01:44.557811 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.557796 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" May 20 19:01:44.558785 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.558770 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-682v4\"" May 20 19:01:44.562903 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.562884 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-24fkf\" (UniqueName: \"kubernetes.io/projected/e31f8582-cd37-4c6f-a313-cd02ca18d675-kube-api-access-24fkf\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:44.562985 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.562912 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e31f8582-cd37-4c6f-a313-cd02ca18d675-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:44.562985 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.562941 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlnx8\" (UniqueName: \"kubernetes.io/projected/9f802bde-bca5-434f-b866-7e5cc08054b9-kube-api-access-hlnx8\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.562985 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.562973 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e31f8582-cd37-4c6f-a313-cd02ca18d675-crio-socket\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:44.563082 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.563037 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9f802bde-bca5-434f-b866-7e5cc08054b9-ca-trust-extracted\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.563116 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.563082 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9f802bde-bca5-434f-b866-7e5cc08054b9-installation-pull-secrets\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.563116 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.563095 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e31f8582-cd37-4c6f-a313-cd02ca18d675-crio-socket\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:44.563116 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.563110 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f802bde-bca5-434f-b866-7e5cc08054b9-trusted-ca\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.563277 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.563142 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9f802bde-bca5-434f-b866-7e5cc08054b9-registry-tls\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.563277 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.563188 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e31f8582-cd37-4c6f-a313-cd02ca18d675-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:44.563277 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.563249 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9f802bde-bca5-434f-b866-7e5cc08054b9-image-registry-private-configuration\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.563277 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.563267 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9f802bde-bca5-434f-b866-7e5cc08054b9-registry-certificates\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.563420 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.563290 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9f802bde-bca5-434f-b866-7e5cc08054b9-bound-sa-token\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.563420 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.563326 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e31f8582-cd37-4c6f-a313-cd02ca18d675-data-volume\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:44.563656 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.563642 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e31f8582-cd37-4c6f-a313-cd02ca18d675-data-volume\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:44.628101 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.628068 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66dc6fb849-rxgv6"] May 20 19:01:44.664302 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.664269 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f802bde-bca5-434f-b866-7e5cc08054b9-trusted-ca\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.664302 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.664303 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9f802bde-bca5-434f-b866-7e5cc08054b9-registry-tls\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.664633 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.664345 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9f802bde-bca5-434f-b866-7e5cc08054b9-image-registry-private-configuration\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.664633 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.664362 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9f802bde-bca5-434f-b866-7e5cc08054b9-registry-certificates\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.664633 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.664379 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9f802bde-bca5-434f-b866-7e5cc08054b9-bound-sa-token\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.664633 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.664423 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hlnx8\" (UniqueName: \"kubernetes.io/projected/9f802bde-bca5-434f-b866-7e5cc08054b9-kube-api-access-hlnx8\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.664949 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.664678 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9f802bde-bca5-434f-b866-7e5cc08054b9-ca-trust-extracted\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.664949 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.664737 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9f802bde-bca5-434f-b866-7e5cc08054b9-installation-pull-secrets\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.665057 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.665039 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9f802bde-bca5-434f-b866-7e5cc08054b9-ca-trust-extracted\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.665391 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.665371 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9f802bde-bca5-434f-b866-7e5cc08054b9-registry-certificates\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.665674 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.665653 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f802bde-bca5-434f-b866-7e5cc08054b9-trusted-ca\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.666990 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.666966 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9f802bde-bca5-434f-b866-7e5cc08054b9-image-registry-private-configuration\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.666990 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.666984 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9f802bde-bca5-434f-b866-7e5cc08054b9-registry-tls\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.667147 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.667061 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9f802bde-bca5-434f-b866-7e5cc08054b9-installation-pull-secrets\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.706989 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.706911 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlnx8\" (UniqueName: \"kubernetes.io/projected/9f802bde-bca5-434f-b866-7e5cc08054b9-kube-api-access-hlnx8\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.722877 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.722839 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9f802bde-bca5-434f-b866-7e5cc08054b9-bound-sa-token\") pod \"image-registry-66dc6fb849-rxgv6\" (UID: \"9f802bde-bca5-434f-b866-7e5cc08054b9\") " pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:44.811005 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:44.810967 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:45.142377 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.142335 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66dc6fb849-rxgv6"] May 20 19:01:45.147721 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:01:45.147693 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f802bde_bca5_434f_b866_7e5cc08054b9.slice/crio-5bf0f481f9ac39632d82a1e0f5e3e1038a07ec9927f7f0b73b2c8c78dac02e22 WatchSource:0}: Error finding container 5bf0f481f9ac39632d82a1e0f5e3e1038a07ec9927f7f0b73b2c8c78dac02e22: Status 404 returned error can't find the container with id 5bf0f481f9ac39632d82a1e0f5e3e1038a07ec9927f7f0b73b2c8c78dac02e22 May 20 19:01:45.274860 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.274827 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 19:01:45.274993 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.274892 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 19:01:45.277018 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.276989 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d601dbc0-665a-4564-833e-bef2c554536c-metrics-tls\") pod \"dns-default-95f7s\" (UID: \"d601dbc0-665a-4564-833e-bef2c554536c\") " pod="openshift-dns/dns-default-95f7s" May 20 19:01:45.277117 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.277052 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4c6b123-27fc-45c3-a22f-6cf99088f182-cert\") pod \"ingress-canary-lhst4\" (UID: \"b4c6b123-27fc-45c3-a22f-6cf99088f182\") " pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 19:01:45.332721 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.332690 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ms74x\"" May 20 19:01:45.340116 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.340081 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-95f7s" May 20 19:01:45.406056 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.405991 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 20 19:01:45.416003 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.415965 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e31f8582-cd37-4c6f-a313-cd02ca18d675-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:45.540737 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.540695 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" event={"ID":"9f802bde-bca5-434f-b866-7e5cc08054b9","Type":"ContainerStarted","Data":"a6a12a56282a3b5f250ccf18e1b741faaa914d96b9b365113b146ba3d99efcc7"} May 20 19:01:45.541194 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.540745 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" event={"ID":"9f802bde-bca5-434f-b866-7e5cc08054b9","Type":"ContainerStarted","Data":"5bf0f481f9ac39632d82a1e0f5e3e1038a07ec9927f7f0b73b2c8c78dac02e22"} May 20 19:01:45.541194 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.540829 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:01:45.541983 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.541959 2575 generic.go:358] "Generic (PLEG): container finished" podID="e41815e6-705b-4f8b-b02d-87b2bb87be6c" containerID="066d77b2870f5f344ac1adcebaef0df4f3f1b7d85147a59b57558972c61f52d7" exitCode=255 May 20 19:01:45.542040 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.541992 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8" event={"ID":"e41815e6-705b-4f8b-b02d-87b2bb87be6c","Type":"ContainerDied","Data":"066d77b2870f5f344ac1adcebaef0df4f3f1b7d85147a59b57558972c61f52d7"} May 20 19:01:45.542265 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.542242 2575 scope.go:117] "RemoveContainer" containerID="066d77b2870f5f344ac1adcebaef0df4f3f1b7d85147a59b57558972c61f52d7" May 20 19:01:45.564049 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:45.564025 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: failed to sync configmap cache: timed out waiting for the condition May 20 19:01:45.564155 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:45.564108 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e31f8582-cd37-4c6f-a313-cd02ca18d675-kube-rbac-proxy-cm podName:e31f8582-cd37-4c6f-a313-cd02ca18d675 nodeName:}" failed. No retries permitted until 2026-05-20 19:01:46.064090421 +0000 UTC m=+163.610049270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e31f8582-cd37-4c6f-a313-cd02ca18d675-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5kdhc" (UID: "e31f8582-cd37-4c6f-a313-cd02ca18d675") : failed to sync configmap cache: timed out waiting for the condition May 20 19:01:45.584308 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.584285 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d28f9\"" May 20 19:01:45.592678 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.592648 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-95f7s"] May 20 19:01:45.595870 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:01:45.595834 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd601dbc0_665a_4564_833e_bef2c554536c.slice/crio-06555ccdb0cfee82284bebce2abd39ced8596d381594e679bc2d1cc4ad9e16dd WatchSource:0}: Error finding container 06555ccdb0cfee82284bebce2abd39ced8596d381594e679bc2d1cc4ad9e16dd: Status 404 returned error can't find the container with id 06555ccdb0cfee82284bebce2abd39ced8596d381594e679bc2d1cc4ad9e16dd May 20 19:01:45.680776 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:45.680728 2575 projected.go:289] Couldn't get configMap openshift-insights/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition May 20 19:01:45.714410 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.714362 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 20 19:01:45.859264 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.859194 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" podStartSLOduration=1.8591782270000001 podStartE2EDuration="1.859178227s" podCreationTimestamp="2026-05-20 19:01:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 19:01:45.77170539 +0000 UTC m=+163.317664244" watchObservedRunningTime="2026-05-20 19:01:45.859178227 +0000 UTC m=+163.405137081" May 20 19:01:45.886615 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:45.886587 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 20 19:01:46.024486 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:46.024407 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 20 19:01:46.031687 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:46.031650 2575 projected.go:194] Error preparing data for projected volume kube-api-access-24fkf for pod openshift-insights/insights-runtime-extractor-5kdhc: failed to sync configmap cache: timed out waiting for the condition May 20 19:01:46.031831 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:01:46.031739 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e31f8582-cd37-4c6f-a313-cd02ca18d675-kube-api-access-24fkf podName:e31f8582-cd37-4c6f-a313-cd02ca18d675 nodeName:}" failed. No retries permitted until 2026-05-20 19:01:46.531723019 +0000 UTC m=+164.077681852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-24fkf" (UniqueName: "kubernetes.io/projected/e31f8582-cd37-4c6f-a313-cd02ca18d675-kube-api-access-24fkf") pod "insights-runtime-extractor-5kdhc" (UID: "e31f8582-cd37-4c6f-a313-cd02ca18d675") : failed to sync configmap cache: timed out waiting for the condition May 20 19:01:46.080998 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:46.080963 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e31f8582-cd37-4c6f-a313-cd02ca18d675-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:46.081507 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:46.081489 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e31f8582-cd37-4c6f-a313-cd02ca18d675-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:46.546186 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:46.546145 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d67dbcc8b-66wm8" event={"ID":"e41815e6-705b-4f8b-b02d-87b2bb87be6c","Type":"ContainerStarted","Data":"f444b98d38815454f5f0a29add3b6730fe29d3d1340d7f5baebdc4b9119a5068"} May 20 19:01:46.547344 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:46.547317 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-95f7s" event={"ID":"d601dbc0-665a-4564-833e-bef2c554536c","Type":"ContainerStarted","Data":"06555ccdb0cfee82284bebce2abd39ced8596d381594e679bc2d1cc4ad9e16dd"} May 20 19:01:46.586121 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:46.586092 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-24fkf\" (UniqueName: \"kubernetes.io/projected/e31f8582-cd37-4c6f-a313-cd02ca18d675-kube-api-access-24fkf\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:46.588729 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:46.588703 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24fkf\" (UniqueName: \"kubernetes.io/projected/e31f8582-cd37-4c6f-a313-cd02ca18d675-kube-api-access-24fkf\") pod \"insights-runtime-extractor-5kdhc\" (UID: \"e31f8582-cd37-4c6f-a313-cd02ca18d675\") " pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:46.833777 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:46.833700 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5kdhc" May 20 19:01:47.034024 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:47.033997 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5kdhc"] May 20 19:01:47.037127 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:01:47.037102 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode31f8582_cd37_4c6f_a313_cd02ca18d675.slice/crio-fec10928a587c3a328ed3bd94dfbf6fbdacc1a54c5bc8105c40865c722642a44 WatchSource:0}: Error finding container fec10928a587c3a328ed3bd94dfbf6fbdacc1a54c5bc8105c40865c722642a44: Status 404 returned error can't find the container with id fec10928a587c3a328ed3bd94dfbf6fbdacc1a54c5bc8105c40865c722642a44 May 20 19:01:47.553095 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:47.553053 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5kdhc" event={"ID":"e31f8582-cd37-4c6f-a313-cd02ca18d675","Type":"ContainerStarted","Data":"4fd77a3b59bd563acb1c706fba727c5be945d0dadc2b2580fcd6914818046311"} May 20 19:01:47.553095 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:47.553095 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5kdhc" event={"ID":"e31f8582-cd37-4c6f-a313-cd02ca18d675","Type":"ContainerStarted","Data":"fec10928a587c3a328ed3bd94dfbf6fbdacc1a54c5bc8105c40865c722642a44"} May 20 19:01:47.554678 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:47.554652 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-95f7s" event={"ID":"d601dbc0-665a-4564-833e-bef2c554536c","Type":"ContainerStarted","Data":"e1840575281bfc40b7ae9631a28b719ea422f8606d7de24b102089d6445ca690"} May 20 19:01:47.554790 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:47.554686 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-95f7s" event={"ID":"d601dbc0-665a-4564-833e-bef2c554536c","Type":"ContainerStarted","Data":"4132c98bb5edace8dbb03a276768c62b4b58a869eaa7beecae2ac0dbb641c120"} May 20 19:01:47.554857 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:47.554836 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-95f7s" May 20 19:01:47.643390 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:47.643340 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-95f7s" podStartSLOduration=129.382213501 podStartE2EDuration="2m10.643324229s" podCreationTimestamp="2026-05-20 18:59:37 +0000 UTC" firstStartedPulling="2026-05-20 19:01:45.597924168 +0000 UTC m=+163.143883006" lastFinishedPulling="2026-05-20 19:01:46.859034899 +0000 UTC m=+164.404993734" observedRunningTime="2026-05-20 19:01:47.641724288 +0000 UTC m=+165.187683143" watchObservedRunningTime="2026-05-20 19:01:47.643324229 +0000 UTC m=+165.189283075" May 20 19:01:48.562356 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:48.562317 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5kdhc" event={"ID":"e31f8582-cd37-4c6f-a313-cd02ca18d675","Type":"ContainerStarted","Data":"739ca54f88bcec8072339fe9e91ef30b48f9779ba92497fb826212e86c62b3cf"} May 20 19:01:49.566585 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:49.566551 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5kdhc" event={"ID":"e31f8582-cd37-4c6f-a313-cd02ca18d675","Type":"ContainerStarted","Data":"203f02e50fce6ffd6901800c5273c188f5c072d4dd6f7400da80b2eea191bfe7"} May 20 19:01:49.616081 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:49.616029 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5kdhc" podStartSLOduration=3.867790113 podStartE2EDuration="5.616015542s" podCreationTimestamp="2026-05-20 19:01:44 +0000 UTC" firstStartedPulling="2026-05-20 19:01:47.085284808 +0000 UTC m=+164.631243641" lastFinishedPulling="2026-05-20 19:01:48.833510234 +0000 UTC m=+166.379469070" observedRunningTime="2026-05-20 19:01:49.615080847 +0000 UTC m=+167.161039702" watchObservedRunningTime="2026-05-20 19:01:49.616015542 +0000 UTC m=+167.161974397" May 20 19:01:52.084879 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:52.084787 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 19:01:52.089328 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:52.089309 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hwgwl\"" May 20 19:01:52.095822 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:52.095790 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lhst4" May 20 19:01:52.243820 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:52.243792 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lhst4"] May 20 19:01:52.247019 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:01:52.246993 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4c6b123_27fc_45c3_a22f_6cf99088f182.slice/crio-ab14f265690af830bb4647da7800d9dc533cf0e9df176049e420aad9caafabf1 WatchSource:0}: Error finding container ab14f265690af830bb4647da7800d9dc533cf0e9df176049e420aad9caafabf1: Status 404 returned error can't find the container with id ab14f265690af830bb4647da7800d9dc533cf0e9df176049e420aad9caafabf1 May 20 19:01:52.574909 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:52.574878 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lhst4" event={"ID":"b4c6b123-27fc-45c3-a22f-6cf99088f182","Type":"ContainerStarted","Data":"ab14f265690af830bb4647da7800d9dc533cf0e9df176049e420aad9caafabf1"} May 20 19:01:53.086591 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.086548 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 19:01:53.619488 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.619449 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-86c7b669dd-sflpt"] May 20 19:01:53.622608 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.622584 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.634364 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.634347 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" May 20 19:01:53.635294 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.635276 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" May 20 19:01:53.635422 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.635358 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" May 20 19:01:53.635422 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.635368 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" May 20 19:01:53.635422 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.635399 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 20 19:01:53.635422 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.635407 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" May 20 19:01:53.635422 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.635414 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gfcwv\"" May 20 19:01:53.649412 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.649397 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 20 19:01:53.653001 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.652981 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c7b669dd-sflpt"] May 20 19:01:53.659792 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.659773 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" May 20 19:01:53.745295 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.745259 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-service-ca\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.745447 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.745304 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-console-config\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.745447 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.745332 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-trusted-ca-bundle\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.745447 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.745360 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dd12c9c-330e-489f-bd3d-0757237726bd-console-oauth-config\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.745447 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.745392 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzmg6\" (UniqueName: \"kubernetes.io/projected/6dd12c9c-330e-489f-bd3d-0757237726bd-kube-api-access-kzmg6\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.745447 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.745438 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-oauth-serving-cert\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.745668 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.745494 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dd12c9c-330e-489f-bd3d-0757237726bd-console-serving-cert\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.846609 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.846563 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-oauth-serving-cert\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.846801 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.846624 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dd12c9c-330e-489f-bd3d-0757237726bd-console-serving-cert\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.846801 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.846686 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-service-ca\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.846801 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.846719 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-console-config\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.846801 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.846744 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-trusted-ca-bundle\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.846801 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.846768 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dd12c9c-330e-489f-bd3d-0757237726bd-console-oauth-config\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.847064 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.846810 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kzmg6\" (UniqueName: \"kubernetes.io/projected/6dd12c9c-330e-489f-bd3d-0757237726bd-kube-api-access-kzmg6\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.847418 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.847388 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-oauth-serving-cert\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.847859 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.847812 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-service-ca\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.847952 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.847871 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-console-config\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.847952 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.847893 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-trusted-ca-bundle\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.849477 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.849458 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dd12c9c-330e-489f-bd3d-0757237726bd-console-serving-cert\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.849634 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.849612 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dd12c9c-330e-489f-bd3d-0757237726bd-console-oauth-config\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.858031 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.858005 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzmg6\" (UniqueName: \"kubernetes.io/projected/6dd12c9c-330e-489f-bd3d-0757237726bd-kube-api-access-kzmg6\") pod \"console-86c7b669dd-sflpt\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:53.934779 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:53.934686 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:01:54.342089 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:54.342053 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c7b669dd-sflpt"] May 20 19:01:54.345423 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:01:54.345396 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd12c9c_330e_489f_bd3d_0757237726bd.slice/crio-4b0caca40450b893c9cb32f5447fa57a125d28474ad0ccf1acd2d32978b01c97 WatchSource:0}: Error finding container 4b0caca40450b893c9cb32f5447fa57a125d28474ad0ccf1acd2d32978b01c97: Status 404 returned error can't find the container with id 4b0caca40450b893c9cb32f5447fa57a125d28474ad0ccf1acd2d32978b01c97 May 20 19:01:54.582152 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:54.582114 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lhst4" event={"ID":"b4c6b123-27fc-45c3-a22f-6cf99088f182","Type":"ContainerStarted","Data":"9eef5c0ca4b44ec677b9bf94555b38b1f0a2457d343f9fcf757be2a33ef0c13d"} May 20 19:01:54.583155 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:54.583130 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c7b669dd-sflpt" event={"ID":"6dd12c9c-330e-489f-bd3d-0757237726bd","Type":"ContainerStarted","Data":"4b0caca40450b893c9cb32f5447fa57a125d28474ad0ccf1acd2d32978b01c97"} May 20 19:01:54.628730 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:54.628678 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lhst4" podStartSLOduration=135.624220036 podStartE2EDuration="2m17.62866335s" podCreationTimestamp="2026-05-20 18:59:37 +0000 UTC" firstStartedPulling="2026-05-20 19:01:52.248883583 +0000 UTC m=+169.794842416" lastFinishedPulling="2026-05-20 19:01:54.253326894 +0000 UTC m=+171.799285730" observedRunningTime="2026-05-20 19:01:54.625366516 +0000 UTC m=+172.171325382" watchObservedRunningTime="2026-05-20 19:01:54.62866335 +0000 UTC m=+172.174622204" May 20 19:01:55.689994 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.689957 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-n56hr"] May 20 19:01:55.693342 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.693309 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.699721 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.699694 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 20 19:01:55.699857 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.699764 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 20 19:01:55.699934 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.699707 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 20 19:01:55.699934 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.699710 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 20 19:01:55.700593 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.700568 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 20 19:01:55.700707 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.700673 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2c9cv\"" May 20 19:01:55.701263 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.701243 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 20 19:01:55.763871 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.763838 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-root\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.764126 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.763892 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-tls\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.764126 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.764017 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-accelerators-collector-config\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.764126 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.764098 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-wtmp\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.764386 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.764157 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-metrics-client-ca\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.764386 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.764186 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlww2\" (UniqueName: \"kubernetes.io/projected/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-kube-api-access-rlww2\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.764386 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.764240 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-sys\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.764386 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.764266 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.764386 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.764294 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-textfile\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.864947 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.864898 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-accelerators-collector-config\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865118 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865075 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-wtmp\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865199 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865125 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-metrics-client-ca\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865199 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865153 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rlww2\" (UniqueName: \"kubernetes.io/projected/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-kube-api-access-rlww2\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865316 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865197 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-sys\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865316 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865284 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-sys\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865316 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865289 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-wtmp\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865460 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865405 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865460 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865445 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-textfile\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865563 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865474 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-root\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865563 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865514 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-tls\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865663 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865604 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-accelerators-collector-config\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865720 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865663 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-root\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865775 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865724 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-textfile\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.865852 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.865833 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-metrics-client-ca\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.867677 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.867659 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.867748 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.867732 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-node-exporter-tls\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:55.879360 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:55.879330 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlww2\" (UniqueName: \"kubernetes.io/projected/5f2ade48-887b-4500-8cbc-2ce26b69f1a7-kube-api-access-rlww2\") pod \"node-exporter-n56hr\" (UID: \"5f2ade48-887b-4500-8cbc-2ce26b69f1a7\") " pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:56.004926 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:56.004837 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n56hr" May 20 19:01:56.892002 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:01:56.891961 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f2ade48_887b_4500_8cbc_2ce26b69f1a7.slice/crio-6558116e84337b63f3cc6464c6d01d3d5ee9f846aca5e513097b092c01c9f5ee WatchSource:0}: Error finding container 6558116e84337b63f3cc6464c6d01d3d5ee9f846aca5e513097b092c01c9f5ee: Status 404 returned error can't find the container with id 6558116e84337b63f3cc6464c6d01d3d5ee9f846aca5e513097b092c01c9f5ee May 20 19:01:57.565710 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:57.565677 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-95f7s" May 20 19:01:57.592350 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:57.592311 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c7b669dd-sflpt" event={"ID":"6dd12c9c-330e-489f-bd3d-0757237726bd","Type":"ContainerStarted","Data":"dd8fcd8776dc97749dcbae9639a5696b463f88b9bf274529d7338a7dbc33cd83"} May 20 19:01:57.593470 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:57.593441 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n56hr" event={"ID":"5f2ade48-887b-4500-8cbc-2ce26b69f1a7","Type":"ContainerStarted","Data":"6558116e84337b63f3cc6464c6d01d3d5ee9f846aca5e513097b092c01c9f5ee"} May 20 19:01:58.597066 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:58.597026 2575 generic.go:358] "Generic (PLEG): container finished" podID="5f2ade48-887b-4500-8cbc-2ce26b69f1a7" containerID="08664d1408711345d90895d7d870c39db1a766662815de78f9dbbbb416bc92f8" exitCode=0 May 20 19:01:58.597508 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:58.597111 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n56hr" event={"ID":"5f2ade48-887b-4500-8cbc-2ce26b69f1a7","Type":"ContainerDied","Data":"08664d1408711345d90895d7d870c39db1a766662815de78f9dbbbb416bc92f8"} May 20 19:01:58.642293 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:58.642248 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-86c7b669dd-sflpt" podStartSLOduration=3.053118665 podStartE2EDuration="5.642232514s" podCreationTimestamp="2026-05-20 19:01:53 +0000 UTC" firstStartedPulling="2026-05-20 19:01:54.347355891 +0000 UTC m=+171.893314737" lastFinishedPulling="2026-05-20 19:01:56.936469742 +0000 UTC m=+174.482428586" observedRunningTime="2026-05-20 19:01:57.629860714 +0000 UTC m=+175.175819570" watchObservedRunningTime="2026-05-20 19:01:58.642232514 +0000 UTC m=+176.188191366" May 20 19:01:59.601511 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:59.601475 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n56hr" event={"ID":"5f2ade48-887b-4500-8cbc-2ce26b69f1a7","Type":"ContainerStarted","Data":"b3bd5ff032cdee1d8cd9078574033d800125929425acf1c9ab775a501f999c08"} May 20 19:01:59.601511 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:59.601511 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n56hr" event={"ID":"5f2ade48-887b-4500-8cbc-2ce26b69f1a7","Type":"ContainerStarted","Data":"122c01d72816e86f8e0994463a1dd1de03984d687943822442a9c075ab4abfc9"} May 20 19:01:59.651376 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:01:59.651323 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-n56hr" podStartSLOduration=3.868460698 podStartE2EDuration="4.651305245s" podCreationTimestamp="2026-05-20 19:01:55 +0000 UTC" firstStartedPulling="2026-05-20 19:01:56.893898449 +0000 UTC m=+174.439857286" lastFinishedPulling="2026-05-20 19:01:57.676743001 +0000 UTC m=+175.222701833" observedRunningTime="2026-05-20 19:01:59.650642436 +0000 UTC m=+177.196601291" watchObservedRunningTime="2026-05-20 19:01:59.651305245 +0000 UTC m=+177.197264097" May 20 19:02:03.935056 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:03.935003 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:02:03.935056 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:03.935073 2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:02:03.939854 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:03.939834 2575 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:02:04.617608 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:04.617581 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:02:04.814931 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:04.814894 2575 patch_prober.go:28] interesting pod/image-registry-66dc6fb849-rxgv6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} May 20 19:02:04.815093 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:04.814953 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" podUID="9f802bde-bca5-434f-b866-7e5cc08054b9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 20 19:02:06.551520 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:06.551488 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66dc6fb849-rxgv6" May 20 19:02:11.173292 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:11.173254 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86c7b669dd-sflpt"] May 20 19:02:36.191519 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.191474 2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-86c7b669dd-sflpt" podUID="6dd12c9c-330e-489f-bd3d-0757237726bd" containerName="console" containerID="cri-o://dd8fcd8776dc97749dcbae9639a5696b463f88b9bf274529d7338a7dbc33cd83" gracePeriod=15 May 20 19:02:36.471141 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.471117 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-86c7b669dd-sflpt_6dd12c9c-330e-489f-bd3d-0757237726bd/console/0.log" May 20 19:02:36.471305 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.471192 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:02:36.599355 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.599314 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-service-ca\") pod \"6dd12c9c-330e-489f-bd3d-0757237726bd\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " May 20 19:02:36.599355 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.599356 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzmg6\" (UniqueName: \"kubernetes.io/projected/6dd12c9c-330e-489f-bd3d-0757237726bd-kube-api-access-kzmg6\") pod \"6dd12c9c-330e-489f-bd3d-0757237726bd\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " May 20 19:02:36.599595 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.599382 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-trusted-ca-bundle\") pod \"6dd12c9c-330e-489f-bd3d-0757237726bd\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " May 20 19:02:36.599595 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.599411 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dd12c9c-330e-489f-bd3d-0757237726bd-console-oauth-config\") pod \"6dd12c9c-330e-489f-bd3d-0757237726bd\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " May 20 19:02:36.599595 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.599441 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dd12c9c-330e-489f-bd3d-0757237726bd-console-serving-cert\") pod \"6dd12c9c-330e-489f-bd3d-0757237726bd\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " May 20 19:02:36.599595 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.599471 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-oauth-serving-cert\") pod \"6dd12c9c-330e-489f-bd3d-0757237726bd\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " May 20 19:02:36.599595 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.599504 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-console-config\") pod \"6dd12c9c-330e-489f-bd3d-0757237726bd\" (UID: \"6dd12c9c-330e-489f-bd3d-0757237726bd\") " May 20 19:02:36.599909 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.599729 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-service-ca" (OuterVolumeSpecName: "service-ca") pod "6dd12c9c-330e-489f-bd3d-0757237726bd" (UID: "6dd12c9c-330e-489f-bd3d-0757237726bd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 20 19:02:36.599909 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.599785 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6dd12c9c-330e-489f-bd3d-0757237726bd" (UID: "6dd12c9c-330e-489f-bd3d-0757237726bd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 20 19:02:36.600015 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.599932 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6dd12c9c-330e-489f-bd3d-0757237726bd" (UID: "6dd12c9c-330e-489f-bd3d-0757237726bd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 20 19:02:36.600015 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.599976 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-console-config" (OuterVolumeSpecName: "console-config") pod "6dd12c9c-330e-489f-bd3d-0757237726bd" (UID: "6dd12c9c-330e-489f-bd3d-0757237726bd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 20 19:02:36.601712 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.601682 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dd12c9c-330e-489f-bd3d-0757237726bd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6dd12c9c-330e-489f-bd3d-0757237726bd" (UID: "6dd12c9c-330e-489f-bd3d-0757237726bd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 20 19:02:36.601712 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.601693 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dd12c9c-330e-489f-bd3d-0757237726bd-kube-api-access-kzmg6" (OuterVolumeSpecName: "kube-api-access-kzmg6") pod "6dd12c9c-330e-489f-bd3d-0757237726bd" (UID: "6dd12c9c-330e-489f-bd3d-0757237726bd"). InnerVolumeSpecName "kube-api-access-kzmg6". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 19:02:36.601870 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.601745 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dd12c9c-330e-489f-bd3d-0757237726bd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6dd12c9c-330e-489f-bd3d-0757237726bd" (UID: "6dd12c9c-330e-489f-bd3d-0757237726bd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 20 19:02:36.695013 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.694984 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-86c7b669dd-sflpt_6dd12c9c-330e-489f-bd3d-0757237726bd/console/0.log" May 20 19:02:36.695198 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.695025 2575 generic.go:358] "Generic (PLEG): container finished" podID="6dd12c9c-330e-489f-bd3d-0757237726bd" containerID="dd8fcd8776dc97749dcbae9639a5696b463f88b9bf274529d7338a7dbc33cd83" exitCode=2 May 20 19:02:36.695198 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.695055 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c7b669dd-sflpt" event={"ID":"6dd12c9c-330e-489f-bd3d-0757237726bd","Type":"ContainerDied","Data":"dd8fcd8776dc97749dcbae9639a5696b463f88b9bf274529d7338a7dbc33cd83"} May 20 19:02:36.695198 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.695094 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c7b669dd-sflpt" event={"ID":"6dd12c9c-330e-489f-bd3d-0757237726bd","Type":"ContainerDied","Data":"4b0caca40450b893c9cb32f5447fa57a125d28474ad0ccf1acd2d32978b01c97"} May 20 19:02:36.695198 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.695106 2575 scope.go:117] "RemoveContainer" containerID="dd8fcd8776dc97749dcbae9639a5696b463f88b9bf274529d7338a7dbc33cd83" May 20 19:02:36.695198 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.695110 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c7b669dd-sflpt" May 20 19:02:36.700185 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.700159 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-service-ca\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:02:36.700185 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.700183 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzmg6\" (UniqueName: \"kubernetes.io/projected/6dd12c9c-330e-489f-bd3d-0757237726bd-kube-api-access-kzmg6\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:02:36.700401 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.700195 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-trusted-ca-bundle\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:02:36.700401 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.700225 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dd12c9c-330e-489f-bd3d-0757237726bd-console-oauth-config\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:02:36.700401 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.700238 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dd12c9c-330e-489f-bd3d-0757237726bd-console-serving-cert\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:02:36.700401 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.700246 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-oauth-serving-cert\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:02:36.700401 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.700255 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dd12c9c-330e-489f-bd3d-0757237726bd-console-config\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:02:36.703033 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.703009 2575 scope.go:117] "RemoveContainer" containerID="dd8fcd8776dc97749dcbae9639a5696b463f88b9bf274529d7338a7dbc33cd83" May 20 19:02:36.703415 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:02:36.703393 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd8fcd8776dc97749dcbae9639a5696b463f88b9bf274529d7338a7dbc33cd83\": container with ID starting with dd8fcd8776dc97749dcbae9639a5696b463f88b9bf274529d7338a7dbc33cd83 not found: ID does not exist" containerID="dd8fcd8776dc97749dcbae9639a5696b463f88b9bf274529d7338a7dbc33cd83" May 20 19:02:36.703496 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.703425 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd8fcd8776dc97749dcbae9639a5696b463f88b9bf274529d7338a7dbc33cd83"} err="failed to get container status \"dd8fcd8776dc97749dcbae9639a5696b463f88b9bf274529d7338a7dbc33cd83\": rpc error: code = NotFound desc = could not find container \"dd8fcd8776dc97749dcbae9639a5696b463f88b9bf274529d7338a7dbc33cd83\": container with ID starting with dd8fcd8776dc97749dcbae9639a5696b463f88b9bf274529d7338a7dbc33cd83 not found: ID does not exist" May 20 19:02:36.724765 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.724686 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86c7b669dd-sflpt"] May 20 19:02:36.729024 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:36.728995 2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86c7b669dd-sflpt"] May 20 19:02:37.088992 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:37.088957 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dd12c9c-330e-489f-bd3d-0757237726bd" path="/var/lib/kubelet/pods/6dd12c9c-330e-489f-bd3d-0757237726bd/volumes" May 20 19:02:49.939759 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:49.939716 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" podUID="3ff0165c-767f-431c-8f22-de468176c86e" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 20 19:02:59.940118 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:02:59.940073 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" podUID="3ff0165c-767f-431c-8f22-de468176c86e" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 20 19:03:09.939671 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:09.939628 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" podUID="3ff0165c-767f-431c-8f22-de468176c86e" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 20 19:03:09.940275 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:09.939714 2575 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" May 20 19:03:09.940412 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:09.940387 2575 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"0f4476d49c703ffad55b29fa2776333c58c5b142589665f7be2b2c784475010d"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" containerMessage="Container service-proxy failed liveness probe, will be restarted" May 20 19:03:09.940472 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:09.940441 2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" podUID="3ff0165c-767f-431c-8f22-de468176c86e" containerName="service-proxy" containerID="cri-o://0f4476d49c703ffad55b29fa2776333c58c5b142589665f7be2b2c784475010d" gracePeriod=30 May 20 19:03:10.786709 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:10.786673 2575 generic.go:358] "Generic (PLEG): container finished" podID="3ff0165c-767f-431c-8f22-de468176c86e" containerID="0f4476d49c703ffad55b29fa2776333c58c5b142589665f7be2b2c784475010d" exitCode=2 May 20 19:03:10.786899 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:10.786753 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" event={"ID":"3ff0165c-767f-431c-8f22-de468176c86e","Type":"ContainerDied","Data":"0f4476d49c703ffad55b29fa2776333c58c5b142589665f7be2b2c784475010d"} May 20 19:03:10.786899 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:10.786796 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77cf746c47-tzpln" event={"ID":"3ff0165c-767f-431c-8f22-de468176c86e","Type":"ContainerStarted","Data":"501667f6b63d9cb5059485203396bbc15119d25fd54b9ae775b6407f24c5dbd0"} May 20 19:03:14.889888 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:14.889829 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 19:03:14.892152 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:14.892128 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c283ba7-2436-4f29-b86c-97340c785bee-metrics-certs\") pod \"network-metrics-daemon-84tr8\" (UID: \"8c283ba7-2436-4f29-b86c-97340c785bee\") " pod="openshift-multus/network-metrics-daemon-84tr8" May 20 19:03:14.990337 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:14.990297 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b6xjs\"" May 20 19:03:14.997690 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:14.997663 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-84tr8" May 20 19:03:15.126649 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:15.126617 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-84tr8"] May 20 19:03:15.129631 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:03:15.129602 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c283ba7_2436_4f29_b86c_97340c785bee.slice/crio-aa779493c9aa8715966b42ad650cf6222482434aa464747e4ee4e265cd599c0c WatchSource:0}: Error finding container aa779493c9aa8715966b42ad650cf6222482434aa464747e4ee4e265cd599c0c: Status 404 returned error can't find the container with id aa779493c9aa8715966b42ad650cf6222482434aa464747e4ee4e265cd599c0c May 20 19:03:15.801938 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:15.801900 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-84tr8" event={"ID":"8c283ba7-2436-4f29-b86c-97340c785bee","Type":"ContainerStarted","Data":"aa779493c9aa8715966b42ad650cf6222482434aa464747e4ee4e265cd599c0c"} May 20 19:03:16.810371 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:16.810333 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-84tr8" event={"ID":"8c283ba7-2436-4f29-b86c-97340c785bee","Type":"ContainerStarted","Data":"a5995e9860695f67eb6685d1fbaddd66281afd2513c78266c6bdf92376b9a68c"} May 20 19:03:16.810371 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:16.810370 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-84tr8" event={"ID":"8c283ba7-2436-4f29-b86c-97340c785bee","Type":"ContainerStarted","Data":"4eab95e6af43555cdebf272f9efb395438ea62406615fbacfb46508ef55250ec"} May 20 19:03:16.827758 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:16.827700 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-84tr8" podStartSLOduration=252.849207271 podStartE2EDuration="4m13.827685125s" podCreationTimestamp="2026-05-20 18:59:03 +0000 UTC" firstStartedPulling="2026-05-20 19:03:15.131750948 +0000 UTC m=+252.677709780" lastFinishedPulling="2026-05-20 19:03:16.110228799 +0000 UTC m=+253.656187634" observedRunningTime="2026-05-20 19:03:16.826526639 +0000 UTC m=+254.372485494" watchObservedRunningTime="2026-05-20 19:03:16.827685125 +0000 UTC m=+254.373643980" May 20 19:03:19.938034 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:19.938001 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6895ffb49b-75r8q"] May 20 19:03:19.938428 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:19.938240 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dd12c9c-330e-489f-bd3d-0757237726bd" containerName="console" May 20 19:03:19.938428 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:19.938252 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dd12c9c-330e-489f-bd3d-0757237726bd" containerName="console" May 20 19:03:19.938428 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:19.938301 2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="6dd12c9c-330e-489f-bd3d-0757237726bd" containerName="console" May 20 19:03:19.941183 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:19.941161 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:19.945051 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:19.945030 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" May 20 19:03:19.945190 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:19.945030 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-996q2\"" May 20 19:03:19.945384 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:19.945365 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" May 20 19:03:19.945455 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:19.945439 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" May 20 19:03:19.946340 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:19.946291 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" May 20 19:03:19.946604 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:19.946442 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" May 20 19:03:19.949944 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:19.949923 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" May 20 19:03:19.953959 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:19.953931 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6895ffb49b-75r8q"] May 20 19:03:20.127805 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.127721 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/67cfd3a3-45b9-49a3-9252-d1dbb354b137-secret-telemeter-client\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.127805 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.127765 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67cfd3a3-45b9-49a3-9252-d1dbb354b137-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.127993 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.127820 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67cfd3a3-45b9-49a3-9252-d1dbb354b137-metrics-client-ca\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.127993 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.127872 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67cfd3a3-45b9-49a3-9252-d1dbb354b137-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.127993 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.127908 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/67cfd3a3-45b9-49a3-9252-d1dbb354b137-telemeter-client-tls\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.127993 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.127931 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2hvm\" (UniqueName: \"kubernetes.io/projected/67cfd3a3-45b9-49a3-9252-d1dbb354b137-kube-api-access-v2hvm\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.127993 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.127955 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/67cfd3a3-45b9-49a3-9252-d1dbb354b137-federate-client-tls\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.127993 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.127985 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67cfd3a3-45b9-49a3-9252-d1dbb354b137-serving-certs-ca-bundle\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.229250 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.229146 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/67cfd3a3-45b9-49a3-9252-d1dbb354b137-secret-telemeter-client\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.229250 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.229195 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67cfd3a3-45b9-49a3-9252-d1dbb354b137-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.229450 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.229253 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67cfd3a3-45b9-49a3-9252-d1dbb354b137-metrics-client-ca\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.229450 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.229290 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67cfd3a3-45b9-49a3-9252-d1dbb354b137-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.229450 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.229331 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/67cfd3a3-45b9-49a3-9252-d1dbb354b137-telemeter-client-tls\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.229450 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.229358 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v2hvm\" (UniqueName: \"kubernetes.io/projected/67cfd3a3-45b9-49a3-9252-d1dbb354b137-kube-api-access-v2hvm\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.229450 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.229387 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/67cfd3a3-45b9-49a3-9252-d1dbb354b137-federate-client-tls\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.229694 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.229463 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67cfd3a3-45b9-49a3-9252-d1dbb354b137-serving-certs-ca-bundle\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.230177 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.230150 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67cfd3a3-45b9-49a3-9252-d1dbb354b137-metrics-client-ca\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.230406 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.230152 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67cfd3a3-45b9-49a3-9252-d1dbb354b137-serving-certs-ca-bundle\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.230406 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.230176 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67cfd3a3-45b9-49a3-9252-d1dbb354b137-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.231848 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.231825 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/67cfd3a3-45b9-49a3-9252-d1dbb354b137-telemeter-client-tls\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.231978 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.231954 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/67cfd3a3-45b9-49a3-9252-d1dbb354b137-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.232023 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.232004 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/67cfd3a3-45b9-49a3-9252-d1dbb354b137-secret-telemeter-client\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.232023 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.232015 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/67cfd3a3-45b9-49a3-9252-d1dbb354b137-federate-client-tls\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.239609 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.239586 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2hvm\" (UniqueName: \"kubernetes.io/projected/67cfd3a3-45b9-49a3-9252-d1dbb354b137-kube-api-access-v2hvm\") pod \"telemeter-client-6895ffb49b-75r8q\" (UID: \"67cfd3a3-45b9-49a3-9252-d1dbb354b137\") " pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.250418 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.250388 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" May 20 19:03:20.392275 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.389608 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6895ffb49b-75r8q"] May 20 19:03:20.821644 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:20.821609 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" event={"ID":"67cfd3a3-45b9-49a3-9252-d1dbb354b137","Type":"ContainerStarted","Data":"a14aebcdb6b2c55a9d4f4e63eec72612a461b119678043d6af5526c4abd48ed0"} May 20 19:03:23.831691 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:23.831664 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" event={"ID":"67cfd3a3-45b9-49a3-9252-d1dbb354b137","Type":"ContainerStarted","Data":"44ef791cf8855c6de5d378741a7a2d0f018bd874f3242d8ffbbb0783e047e488"} May 20 19:03:24.835428 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:24.835392 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" event={"ID":"67cfd3a3-45b9-49a3-9252-d1dbb354b137","Type":"ContainerStarted","Data":"c4c319dbc0bdb638e06c5b4dd529de89cc4ad86b141c70d22c108a771f991d06"} May 20 19:03:24.835428 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:24.835431 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" event={"ID":"67cfd3a3-45b9-49a3-9252-d1dbb354b137","Type":"ContainerStarted","Data":"0da7bc343b0024beed1268244e9251d1b871b61c1b9db7a440a2dda291f1c9e3"} May 20 19:03:24.858745 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:03:24.858692 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6895ffb49b-75r8q" podStartSLOduration=2.433199299 podStartE2EDuration="5.858676857s" podCreationTimestamp="2026-05-20 19:03:19 +0000 UTC" firstStartedPulling="2026-05-20 19:03:20.394973917 +0000 UTC m=+257.940932761" lastFinishedPulling="2026-05-20 19:03:23.820451487 +0000 UTC m=+261.366410319" observedRunningTime="2026-05-20 19:03:24.857456096 +0000 UTC m=+262.403414952" watchObservedRunningTime="2026-05-20 19:03:24.858676857 +0000 UTC m=+262.404635712" May 20 19:04:02.966400 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:04:02.966369 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:04:02.975491 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:04:02.975457 2575 kubelet.go:1628] "Image garbage collection succeeded" May 20 19:04:02.976015 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:04:02.975990 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:07:24.038768 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.038734 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-ph5pt"] May 20 19:07:24.041634 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.041614 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-ph5pt" May 20 19:07:24.045434 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.045408 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" May 20 19:07:24.045595 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.045437 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-sdw7s\"" May 20 19:07:24.046579 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.046558 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" May 20 19:07:24.060997 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.060966 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-ph5pt"] May 20 19:07:24.149398 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.149359 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbpk6\" (UniqueName: \"kubernetes.io/projected/c052637a-b7e2-4df3-8c90-9df75448684a-kube-api-access-mbpk6\") pod \"cert-manager-79c8d999ff-ph5pt\" (UID: \"c052637a-b7e2-4df3-8c90-9df75448684a\") " pod="cert-manager/cert-manager-79c8d999ff-ph5pt" May 20 19:07:24.149577 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.149417 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c052637a-b7e2-4df3-8c90-9df75448684a-bound-sa-token\") pod \"cert-manager-79c8d999ff-ph5pt\" (UID: \"c052637a-b7e2-4df3-8c90-9df75448684a\") " pod="cert-manager/cert-manager-79c8d999ff-ph5pt" May 20 19:07:24.250304 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.250267 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mbpk6\" (UniqueName: \"kubernetes.io/projected/c052637a-b7e2-4df3-8c90-9df75448684a-kube-api-access-mbpk6\") pod \"cert-manager-79c8d999ff-ph5pt\" (UID: \"c052637a-b7e2-4df3-8c90-9df75448684a\") " pod="cert-manager/cert-manager-79c8d999ff-ph5pt" May 20 19:07:24.250470 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.250314 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c052637a-b7e2-4df3-8c90-9df75448684a-bound-sa-token\") pod \"cert-manager-79c8d999ff-ph5pt\" (UID: \"c052637a-b7e2-4df3-8c90-9df75448684a\") " pod="cert-manager/cert-manager-79c8d999ff-ph5pt" May 20 19:07:24.260980 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.260944 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c052637a-b7e2-4df3-8c90-9df75448684a-bound-sa-token\") pod \"cert-manager-79c8d999ff-ph5pt\" (UID: \"c052637a-b7e2-4df3-8c90-9df75448684a\") " pod="cert-manager/cert-manager-79c8d999ff-ph5pt" May 20 19:07:24.261251 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.261232 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbpk6\" (UniqueName: \"kubernetes.io/projected/c052637a-b7e2-4df3-8c90-9df75448684a-kube-api-access-mbpk6\") pod \"cert-manager-79c8d999ff-ph5pt\" (UID: \"c052637a-b7e2-4df3-8c90-9df75448684a\") " pod="cert-manager/cert-manager-79c8d999ff-ph5pt" May 20 19:07:24.351735 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.351650 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-ph5pt" May 20 19:07:24.483596 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.483564 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-ph5pt"] May 20 19:07:24.483886 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:07:24.483861 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc052637a_b7e2_4df3_8c90_9df75448684a.slice/crio-f1a3cc327a7f55aa251eb5e1d0ccfcb636c234c99aaea138dd8857bb4f6c0570 WatchSource:0}: Error finding container f1a3cc327a7f55aa251eb5e1d0ccfcb636c234c99aaea138dd8857bb4f6c0570: Status 404 returned error can't find the container with id f1a3cc327a7f55aa251eb5e1d0ccfcb636c234c99aaea138dd8857bb4f6c0570 May 20 19:07:24.485598 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:24.485581 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 20 19:07:25.453063 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:25.453015 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-ph5pt" event={"ID":"c052637a-b7e2-4df3-8c90-9df75448684a","Type":"ContainerStarted","Data":"f1a3cc327a7f55aa251eb5e1d0ccfcb636c234c99aaea138dd8857bb4f6c0570"} May 20 19:07:27.460942 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:27.460907 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-ph5pt" event={"ID":"c052637a-b7e2-4df3-8c90-9df75448684a","Type":"ContainerStarted","Data":"ca91fd251e685dc5e3af11a5fc85814ac5f74bca6058a5d38e075f2db4c45b4d"} May 20 19:07:27.480468 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:27.480417 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-ph5pt" podStartSLOduration=1.070356404 podStartE2EDuration="3.480404067s" podCreationTimestamp="2026-05-20 19:07:24 +0000 UTC" firstStartedPulling="2026-05-20 19:07:24.485705072 +0000 UTC m=+502.031663908" lastFinishedPulling="2026-05-20 19:07:26.895752734 +0000 UTC m=+504.441711571" observedRunningTime="2026-05-20 19:07:27.479489119 +0000 UTC m=+505.025447973" watchObservedRunningTime="2026-05-20 19:07:27.480404067 +0000 UTC m=+505.026362921" May 20 19:07:41.755590 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:41.755550 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks"] May 20 19:07:41.757581 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:41.757563 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:41.763010 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:41.762955 2575 status_manager.go:895] "Failed to get status for pod" podUID="8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2" pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" err="pods \"opendatahub-operator-controller-manager-6764c5ff7d-g9jks\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" May 20 19:07:41.763010 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:07:41.762981 2575 reflector.go:200] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-webhook-cert\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" type="*v1.Secret" May 20 19:07:41.763010 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:07:41.762993 2575 reflector.go:200] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-dockercfg-cj7wr\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cj7wr\"" type="*v1.Secret" May 20 19:07:41.763010 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:07:41.762984 2575 reflector.go:200] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-service-cert\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" May 20 19:07:41.763010 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:07:41.762992 2575 reflector.go:200] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" May 20 19:07:41.763311 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:07:41.763068 2575 reflector.go:200] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" May 20 19:07:41.768773 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:41.768750 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-6764c5ff7d-g9jks\" (UID: \"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2\") " pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:41.768875 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:41.768804 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6764c5ff7d-g9jks\" (UID: \"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2\") " pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:41.768875 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:41.768850 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4m6f\" (UniqueName: \"kubernetes.io/projected/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-kube-api-access-c4m6f\") pod \"opendatahub-operator-controller-manager-6764c5ff7d-g9jks\" (UID: \"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2\") " pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:41.786927 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:41.786897 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks"] May 20 19:07:41.869275 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:41.869241 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-6764c5ff7d-g9jks\" (UID: \"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2\") " pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:41.869456 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:41.869300 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6764c5ff7d-g9jks\" (UID: \"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2\") " pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:41.869456 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:41.869318 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c4m6f\" (UniqueName: \"kubernetes.io/projected/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-kube-api-access-c4m6f\") pod \"opendatahub-operator-controller-manager-6764c5ff7d-g9jks\" (UID: \"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2\") " pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:42.586369 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:42.586334 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cj7wr\"" May 20 19:07:42.870547 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:07:42.870461 2575 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition May 20 19:07:42.870547 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:07:42.870541 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-apiservice-cert podName:8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2 nodeName:}" failed. No retries permitted until 2026-05-20 19:07:43.370523447 +0000 UTC m=+520.916482285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "apiservice-cert" (UniqueName: "kubernetes.io/secret/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-apiservice-cert") pod "opendatahub-operator-controller-manager-6764c5ff7d-g9jks" (UID: "8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2") : failed to sync secret cache: timed out waiting for the condition May 20 19:07:42.870927 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:07:42.870471 2575 secret.go:189] Couldn't get secret opendatahub/opendatahub-operator-controller-manager-service-cert: failed to sync secret cache: timed out waiting for the condition May 20 19:07:42.870927 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:07:42.870628 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-webhook-cert podName:8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2 nodeName:}" failed. No retries permitted until 2026-05-20 19:07:43.370614226 +0000 UTC m=+520.916573059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-webhook-cert") pod "opendatahub-operator-controller-manager-6764c5ff7d-g9jks" (UID: "8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2") : failed to sync secret cache: timed out waiting for the condition May 20 19:07:42.916075 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:07:42.916029 2575 projected.go:289] Couldn't get configMap opendatahub/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition May 20 19:07:43.005868 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:43.005830 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 20 19:07:43.006813 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:07:43.006795 2575 projected.go:194] Error preparing data for projected volume kube-api-access-c4m6f for pod opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks: failed to sync configmap cache: timed out waiting for the condition May 20 19:07:43.006883 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:07:43.006859 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-kube-api-access-c4m6f podName:8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2 nodeName:}" failed. No retries permitted until 2026-05-20 19:07:43.506843679 +0000 UTC m=+521.052802512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-c4m6f" (UniqueName: "kubernetes.io/projected/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-kube-api-access-c4m6f") pod "opendatahub-operator-controller-manager-6764c5ff7d-g9jks" (UID: "8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2") : failed to sync configmap cache: timed out waiting for the condition May 20 19:07:43.017578 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:43.017549 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 20 19:07:43.202776 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:43.202695 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 20 19:07:43.308121 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:43.308087 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 20 19:07:43.381345 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:43.381302 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-6764c5ff7d-g9jks\" (UID: \"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2\") " pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:43.381549 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:43.381369 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6764c5ff7d-g9jks\" (UID: \"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2\") " pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:43.383743 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:43.383717 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6764c5ff7d-g9jks\" (UID: \"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2\") " pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:43.383839 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:43.383756 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-6764c5ff7d-g9jks\" (UID: \"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2\") " pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:43.583182 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:43.583140 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c4m6f\" (UniqueName: \"kubernetes.io/projected/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-kube-api-access-c4m6f\") pod \"opendatahub-operator-controller-manager-6764c5ff7d-g9jks\" (UID: \"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2\") " pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:43.585745 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:43.585725 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4m6f\" (UniqueName: \"kubernetes.io/projected/8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2-kube-api-access-c4m6f\") pod \"opendatahub-operator-controller-manager-6764c5ff7d-g9jks\" (UID: \"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2\") " pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:43.867533 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:43.867431 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:44.007381 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:44.007354 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks"] May 20 19:07:44.009601 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:07:44.009578 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ecf7c0c_4e13_4eaf_a5f5_85f05a7a10d2.slice/crio-4faa064d161f118c32b0976b20273bec05efb1bd934b198d00a361e34bfe27f6 WatchSource:0}: Error finding container 4faa064d161f118c32b0976b20273bec05efb1bd934b198d00a361e34bfe27f6: Status 404 returned error can't find the container with id 4faa064d161f118c32b0976b20273bec05efb1bd934b198d00a361e34bfe27f6 May 20 19:07:44.506106 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:44.506053 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" event={"ID":"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2","Type":"ContainerStarted","Data":"4faa064d161f118c32b0976b20273bec05efb1bd934b198d00a361e34bfe27f6"} May 20 19:07:46.513189 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:46.513147 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" event={"ID":"8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2","Type":"ContainerStarted","Data":"65cdf47a0df0ce53635c797d016575ab9a092b2bf1758dd18244ac7e7014d326"} May 20 19:07:46.513570 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:46.513238 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:07:46.548312 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:46.548191 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" podStartSLOduration=3.264526908 podStartE2EDuration="5.548175794s" podCreationTimestamp="2026-05-20 19:07:41 +0000 UTC" firstStartedPulling="2026-05-20 19:07:44.011328296 +0000 UTC m=+521.557287129" lastFinishedPulling="2026-05-20 19:07:46.294977178 +0000 UTC m=+523.840936015" observedRunningTime="2026-05-20 19:07:46.546089663 +0000 UTC m=+524.092048517" watchObservedRunningTime="2026-05-20 19:07:46.548175794 +0000 UTC m=+524.094134650" May 20 19:07:57.519005 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:07:57.518922 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6764c5ff7d-g9jks" May 20 19:08:02.455893 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.455847 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8958bfff9-757xw"] May 20 19:08:02.458083 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.458059 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" May 20 19:08:02.462144 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.462125 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 20 19:08:02.462144 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.462138 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" May 20 19:08:02.463157 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.463143 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 20 19:08:02.463298 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.463280 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qgbkc\"" May 20 19:08:02.463298 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.463291 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" May 20 19:08:02.480913 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.480889 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8958bfff9-757xw"] May 20 19:08:02.527887 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.527846 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a227a7b3-34e3-4f03-bf21-67e0de1c547e-tls-certs\") pod \"kube-auth-proxy-8958bfff9-757xw\" (UID: \"a227a7b3-34e3-4f03-bf21-67e0de1c547e\") " pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" May 20 19:08:02.528069 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.527946 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a227a7b3-34e3-4f03-bf21-67e0de1c547e-tmp\") pod \"kube-auth-proxy-8958bfff9-757xw\" (UID: \"a227a7b3-34e3-4f03-bf21-67e0de1c547e\") " pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" May 20 19:08:02.528069 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.527967 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvmd9\" (UniqueName: \"kubernetes.io/projected/a227a7b3-34e3-4f03-bf21-67e0de1c547e-kube-api-access-nvmd9\") pod \"kube-auth-proxy-8958bfff9-757xw\" (UID: \"a227a7b3-34e3-4f03-bf21-67e0de1c547e\") " pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" May 20 19:08:02.628754 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.628714 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a227a7b3-34e3-4f03-bf21-67e0de1c547e-tls-certs\") pod \"kube-auth-proxy-8958bfff9-757xw\" (UID: \"a227a7b3-34e3-4f03-bf21-67e0de1c547e\") " pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" May 20 19:08:02.628940 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.628785 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a227a7b3-34e3-4f03-bf21-67e0de1c547e-tmp\") pod \"kube-auth-proxy-8958bfff9-757xw\" (UID: \"a227a7b3-34e3-4f03-bf21-67e0de1c547e\") " pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" May 20 19:08:02.628940 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.628806 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nvmd9\" (UniqueName: \"kubernetes.io/projected/a227a7b3-34e3-4f03-bf21-67e0de1c547e-kube-api-access-nvmd9\") pod \"kube-auth-proxy-8958bfff9-757xw\" (UID: \"a227a7b3-34e3-4f03-bf21-67e0de1c547e\") " pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" May 20 19:08:02.630997 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.630960 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a227a7b3-34e3-4f03-bf21-67e0de1c547e-tmp\") pod \"kube-auth-proxy-8958bfff9-757xw\" (UID: \"a227a7b3-34e3-4f03-bf21-67e0de1c547e\") " pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" May 20 19:08:02.631178 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.631159 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a227a7b3-34e3-4f03-bf21-67e0de1c547e-tls-certs\") pod \"kube-auth-proxy-8958bfff9-757xw\" (UID: \"a227a7b3-34e3-4f03-bf21-67e0de1c547e\") " pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" May 20 19:08:02.643726 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.643701 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvmd9\" (UniqueName: \"kubernetes.io/projected/a227a7b3-34e3-4f03-bf21-67e0de1c547e-kube-api-access-nvmd9\") pod \"kube-auth-proxy-8958bfff9-757xw\" (UID: \"a227a7b3-34e3-4f03-bf21-67e0de1c547e\") " pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" May 20 19:08:02.766845 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.766759 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" May 20 19:08:02.913426 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:02.913394 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8958bfff9-757xw"] May 20 19:08:02.917089 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:08:02.917063 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda227a7b3_34e3_4f03_bf21_67e0de1c547e.slice/crio-d19c6e5d1e63a9706cbebc188492002aa675fe001e99f250bcdbb6314e110c4c WatchSource:0}: Error finding container d19c6e5d1e63a9706cbebc188492002aa675fe001e99f250bcdbb6314e110c4c: Status 404 returned error can't find the container with id d19c6e5d1e63a9706cbebc188492002aa675fe001e99f250bcdbb6314e110c4c May 20 19:08:03.564962 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:03.564924 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" event={"ID":"a227a7b3-34e3-4f03-bf21-67e0de1c547e","Type":"ContainerStarted","Data":"d19c6e5d1e63a9706cbebc188492002aa675fe001e99f250bcdbb6314e110c4c"} May 20 19:08:05.215753 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:05.215715 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-qlgtq"] May 20 19:08:05.218144 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:05.218124 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:05.220853 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:05.220830 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-8ctxz\"" May 20 19:08:05.220983 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:05.220870 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" May 20 19:08:05.229318 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:05.229293 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-qlgtq"] May 20 19:08:05.250920 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:05.250886 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvbqc\" (UniqueName: \"kubernetes.io/projected/13823e31-0602-45a5-bb05-e980ef446c1c-kube-api-access-bvbqc\") pod \"odh-model-controller-858dbf95b8-qlgtq\" (UID: \"13823e31-0602-45a5-bb05-e980ef446c1c\") " pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:05.251075 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:05.250941 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13823e31-0602-45a5-bb05-e980ef446c1c-cert\") pod \"odh-model-controller-858dbf95b8-qlgtq\" (UID: \"13823e31-0602-45a5-bb05-e980ef446c1c\") " pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:05.352217 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:05.352172 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13823e31-0602-45a5-bb05-e980ef446c1c-cert\") pod \"odh-model-controller-858dbf95b8-qlgtq\" (UID: \"13823e31-0602-45a5-bb05-e980ef446c1c\") " pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:05.352414 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:05.352289 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bvbqc\" (UniqueName: \"kubernetes.io/projected/13823e31-0602-45a5-bb05-e980ef446c1c-kube-api-access-bvbqc\") pod \"odh-model-controller-858dbf95b8-qlgtq\" (UID: \"13823e31-0602-45a5-bb05-e980ef446c1c\") " pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:05.352414 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:08:05.352347 2575 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found May 20 19:08:05.352512 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:08:05.352435 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13823e31-0602-45a5-bb05-e980ef446c1c-cert podName:13823e31-0602-45a5-bb05-e980ef446c1c nodeName:}" failed. No retries permitted until 2026-05-20 19:08:05.85241436 +0000 UTC m=+543.398373226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/13823e31-0602-45a5-bb05-e980ef446c1c-cert") pod "odh-model-controller-858dbf95b8-qlgtq" (UID: "13823e31-0602-45a5-bb05-e980ef446c1c") : secret "odh-model-controller-webhook-cert" not found May 20 19:08:05.363089 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:05.363057 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvbqc\" (UniqueName: \"kubernetes.io/projected/13823e31-0602-45a5-bb05-e980ef446c1c-kube-api-access-bvbqc\") pod \"odh-model-controller-858dbf95b8-qlgtq\" (UID: \"13823e31-0602-45a5-bb05-e980ef446c1c\") " pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:05.856835 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:05.856795 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13823e31-0602-45a5-bb05-e980ef446c1c-cert\") pod \"odh-model-controller-858dbf95b8-qlgtq\" (UID: \"13823e31-0602-45a5-bb05-e980ef446c1c\") " pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:05.857007 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:08:05.856962 2575 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found May 20 19:08:05.857075 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:08:05.857064 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13823e31-0602-45a5-bb05-e980ef446c1c-cert podName:13823e31-0602-45a5-bb05-e980ef446c1c nodeName:}" failed. No retries permitted until 2026-05-20 19:08:06.857041483 +0000 UTC m=+544.403000321 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/13823e31-0602-45a5-bb05-e980ef446c1c-cert") pod "odh-model-controller-858dbf95b8-qlgtq" (UID: "13823e31-0602-45a5-bb05-e980ef446c1c") : secret "odh-model-controller-webhook-cert" not found May 20 19:08:06.474592 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:06.474566 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" May 20 19:08:06.574363 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:06.574325 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" event={"ID":"a227a7b3-34e3-4f03-bf21-67e0de1c547e","Type":"ContainerStarted","Data":"773e2f07f6e042c8a7447f8ee56f3b5cd15f127baff20b023d02e08d18b7bb76"} May 20 19:08:06.617088 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:06.617037 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8958bfff9-757xw" podStartSLOduration=1.065630306 podStartE2EDuration="4.617011692s" podCreationTimestamp="2026-05-20 19:08:02 +0000 UTC" firstStartedPulling="2026-05-20 19:08:02.918650903 +0000 UTC m=+540.464609736" lastFinishedPulling="2026-05-20 19:08:06.470032278 +0000 UTC m=+544.015991122" observedRunningTime="2026-05-20 19:08:06.61453607 +0000 UTC m=+544.160494924" watchObservedRunningTime="2026-05-20 19:08:06.617011692 +0000 UTC m=+544.162970547" May 20 19:08:06.865962 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:06.865932 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13823e31-0602-45a5-bb05-e980ef446c1c-cert\") pod \"odh-model-controller-858dbf95b8-qlgtq\" (UID: \"13823e31-0602-45a5-bb05-e980ef446c1c\") " pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:06.868291 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:06.868268 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13823e31-0602-45a5-bb05-e980ef446c1c-cert\") pod \"odh-model-controller-858dbf95b8-qlgtq\" (UID: \"13823e31-0602-45a5-bb05-e980ef446c1c\") " pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:07.028198 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:07.028141 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:07.166049 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:07.165973 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-qlgtq"] May 20 19:08:07.169432 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:08:07.169401 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13823e31_0602_45a5_bb05_e980ef446c1c.slice/crio-97babe72f937cc2565a125de0b0d7d266edb7e8c29e19945371db604f559d094 WatchSource:0}: Error finding container 97babe72f937cc2565a125de0b0d7d266edb7e8c29e19945371db604f559d094: Status 404 returned error can't find the container with id 97babe72f937cc2565a125de0b0d7d266edb7e8c29e19945371db604f559d094 May 20 19:08:07.578004 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:07.577972 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" event={"ID":"13823e31-0602-45a5-bb05-e980ef446c1c","Type":"ContainerStarted","Data":"97babe72f937cc2565a125de0b0d7d266edb7e8c29e19945371db604f559d094"} May 20 19:08:10.592498 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:10.592459 2575 generic.go:358] "Generic (PLEG): container finished" podID="13823e31-0602-45a5-bb05-e980ef446c1c" containerID="cabda7d91119b167999608cd3bf3d41cdcd9325ec8cfa2fa627aca00f331b81b" exitCode=1 May 20 19:08:10.592864 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:10.592522 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" event={"ID":"13823e31-0602-45a5-bb05-e980ef446c1c","Type":"ContainerDied","Data":"cabda7d91119b167999608cd3bf3d41cdcd9325ec8cfa2fa627aca00f331b81b"} May 20 19:08:10.592864 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:10.592731 2575 scope.go:117] "RemoveContainer" containerID="cabda7d91119b167999608cd3bf3d41cdcd9325ec8cfa2fa627aca00f331b81b" May 20 19:08:10.772550 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:10.772514 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-6lwqb"] May 20 19:08:10.775233 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:10.775195 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" May 20 19:08:10.778021 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:10.778000 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-gm7gx\"" May 20 19:08:10.780120 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:10.780099 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" May 20 19:08:10.791274 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:10.791248 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-6lwqb"] May 20 19:08:10.901218 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:10.901180 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4520054d-c70f-41fe-8011-220449c622b3-cert\") pod \"kserve-controller-manager-856948b99f-6lwqb\" (UID: \"4520054d-c70f-41fe-8011-220449c622b3\") " pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" May 20 19:08:10.901330 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:10.901311 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcgs5\" (UniqueName: \"kubernetes.io/projected/4520054d-c70f-41fe-8011-220449c622b3-kube-api-access-kcgs5\") pod \"kserve-controller-manager-856948b99f-6lwqb\" (UID: \"4520054d-c70f-41fe-8011-220449c622b3\") " pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" May 20 19:08:11.001835 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:11.001795 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kcgs5\" (UniqueName: \"kubernetes.io/projected/4520054d-c70f-41fe-8011-220449c622b3-kube-api-access-kcgs5\") pod \"kserve-controller-manager-856948b99f-6lwqb\" (UID: \"4520054d-c70f-41fe-8011-220449c622b3\") " pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" May 20 19:08:11.001835 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:11.001841 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4520054d-c70f-41fe-8011-220449c622b3-cert\") pod \"kserve-controller-manager-856948b99f-6lwqb\" (UID: \"4520054d-c70f-41fe-8011-220449c622b3\") " pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" May 20 19:08:11.002058 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:08:11.001975 2575 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found May 20 19:08:11.002058 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:08:11.002028 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4520054d-c70f-41fe-8011-220449c622b3-cert podName:4520054d-c70f-41fe-8011-220449c622b3 nodeName:}" failed. No retries permitted until 2026-05-20 19:08:11.502011942 +0000 UTC m=+549.047970774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4520054d-c70f-41fe-8011-220449c622b3-cert") pod "kserve-controller-manager-856948b99f-6lwqb" (UID: "4520054d-c70f-41fe-8011-220449c622b3") : secret "kserve-webhook-server-cert" not found May 20 19:08:11.013437 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:11.013410 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcgs5\" (UniqueName: \"kubernetes.io/projected/4520054d-c70f-41fe-8011-220449c622b3-kube-api-access-kcgs5\") pod \"kserve-controller-manager-856948b99f-6lwqb\" (UID: \"4520054d-c70f-41fe-8011-220449c622b3\") " pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" May 20 19:08:11.506528 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:11.506492 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4520054d-c70f-41fe-8011-220449c622b3-cert\") pod \"kserve-controller-manager-856948b99f-6lwqb\" (UID: \"4520054d-c70f-41fe-8011-220449c622b3\") " pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" May 20 19:08:11.508938 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:11.508912 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4520054d-c70f-41fe-8011-220449c622b3-cert\") pod \"kserve-controller-manager-856948b99f-6lwqb\" (UID: \"4520054d-c70f-41fe-8011-220449c622b3\") " pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" May 20 19:08:11.596198 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:11.596166 2575 generic.go:358] "Generic (PLEG): container finished" podID="13823e31-0602-45a5-bb05-e980ef446c1c" containerID="f5e050be3f36dddf94b47ac81571fb850778197a63d9325915cfdc57c6a25154" exitCode=1 May 20 19:08:11.596622 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:11.596255 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" event={"ID":"13823e31-0602-45a5-bb05-e980ef446c1c","Type":"ContainerDied","Data":"f5e050be3f36dddf94b47ac81571fb850778197a63d9325915cfdc57c6a25154"} May 20 19:08:11.596622 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:11.596303 2575 scope.go:117] "RemoveContainer" containerID="cabda7d91119b167999608cd3bf3d41cdcd9325ec8cfa2fa627aca00f331b81b" May 20 19:08:11.596622 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:11.596492 2575 scope.go:117] "RemoveContainer" containerID="f5e050be3f36dddf94b47ac81571fb850778197a63d9325915cfdc57c6a25154" May 20 19:08:11.596725 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:08:11.596698 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-qlgtq_opendatahub(13823e31-0602-45a5-bb05-e980ef446c1c)\"" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" podUID="13823e31-0602-45a5-bb05-e980ef446c1c" May 20 19:08:11.687649 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:11.687615 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" May 20 19:08:11.840489 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:11.840457 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-6lwqb"] May 20 19:08:11.843398 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:08:11.843356 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4520054d_c70f_41fe_8011_220449c622b3.slice/crio-1726e0c3bb9c32acaf5f00d54ece5ce20ae1fb9f429447b47811e56049f82f04 WatchSource:0}: Error finding container 1726e0c3bb9c32acaf5f00d54ece5ce20ae1fb9f429447b47811e56049f82f04: Status 404 returned error can't find the container with id 1726e0c3bb9c32acaf5f00d54ece5ce20ae1fb9f429447b47811e56049f82f04 May 20 19:08:12.600130 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:12.600091 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" event={"ID":"4520054d-c70f-41fe-8011-220449c622b3","Type":"ContainerStarted","Data":"1726e0c3bb9c32acaf5f00d54ece5ce20ae1fb9f429447b47811e56049f82f04"} May 20 19:08:12.601635 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:12.601616 2575 scope.go:117] "RemoveContainer" containerID="f5e050be3f36dddf94b47ac81571fb850778197a63d9325915cfdc57c6a25154" May 20 19:08:12.601792 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:08:12.601776 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-qlgtq_opendatahub(13823e31-0602-45a5-bb05-e980ef446c1c)\"" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" podUID="13823e31-0602-45a5-bb05-e980ef446c1c" May 20 19:08:15.610269 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:15.610226 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" event={"ID":"4520054d-c70f-41fe-8011-220449c622b3","Type":"ContainerStarted","Data":"bae03af0374b313559ab43d9e5d3ec2f738f4b6b3996d972d19ebeac600e92d4"} May 20 19:08:15.610656 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:15.610339 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" May 20 19:08:15.729169 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:15.729112 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" podStartSLOduration=2.867608678 podStartE2EDuration="5.729095706s" podCreationTimestamp="2026-05-20 19:08:10 +0000 UTC" firstStartedPulling="2026-05-20 19:08:11.844800293 +0000 UTC m=+549.390759126" lastFinishedPulling="2026-05-20 19:08:14.706287309 +0000 UTC m=+552.252246154" observedRunningTime="2026-05-20 19:08:15.728889104 +0000 UTC m=+553.274847960" watchObservedRunningTime="2026-05-20 19:08:15.729095706 +0000 UTC m=+553.275054561" May 20 19:08:17.028494 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:17.028446 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:17.028884 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:17.028867 2575 scope.go:117] "RemoveContainer" containerID="f5e050be3f36dddf94b47ac81571fb850778197a63d9325915cfdc57c6a25154" May 20 19:08:17.029085 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:08:17.029067 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-qlgtq_opendatahub(13823e31-0602-45a5-bb05-e980ef446c1c)\"" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" podUID="13823e31-0602-45a5-bb05-e980ef446c1c" May 20 19:08:23.753813 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:23.753774 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f"] May 20 19:08:23.756677 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:23.756658 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f" May 20 19:08:23.760829 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:23.760804 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" May 20 19:08:23.761231 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:23.761195 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" May 20 19:08:23.761671 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:23.761653 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-vdqvw\"" May 20 19:08:23.831121 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:23.831092 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f"] May 20 19:08:23.909406 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:23.909376 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/08ec4678-017b-451e-93d4-c570b48b5e75-operator-config\") pod \"servicemesh-operator3-55f49c5f94-dnk6f\" (UID: \"08ec4678-017b-451e-93d4-c570b48b5e75\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f" May 20 19:08:23.909585 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:23.909424 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zplql\" (UniqueName: \"kubernetes.io/projected/08ec4678-017b-451e-93d4-c570b48b5e75-kube-api-access-zplql\") pod \"servicemesh-operator3-55f49c5f94-dnk6f\" (UID: \"08ec4678-017b-451e-93d4-c570b48b5e75\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f" May 20 19:08:24.010132 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:24.010024 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/08ec4678-017b-451e-93d4-c570b48b5e75-operator-config\") pod \"servicemesh-operator3-55f49c5f94-dnk6f\" (UID: \"08ec4678-017b-451e-93d4-c570b48b5e75\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f" May 20 19:08:24.010132 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:24.010088 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zplql\" (UniqueName: \"kubernetes.io/projected/08ec4678-017b-451e-93d4-c570b48b5e75-kube-api-access-zplql\") pod \"servicemesh-operator3-55f49c5f94-dnk6f\" (UID: \"08ec4678-017b-451e-93d4-c570b48b5e75\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f" May 20 19:08:24.012519 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:24.012489 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/08ec4678-017b-451e-93d4-c570b48b5e75-operator-config\") pod \"servicemesh-operator3-55f49c5f94-dnk6f\" (UID: \"08ec4678-017b-451e-93d4-c570b48b5e75\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f" May 20 19:08:24.038414 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:24.038378 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zplql\" (UniqueName: \"kubernetes.io/projected/08ec4678-017b-451e-93d4-c570b48b5e75-kube-api-access-zplql\") pod \"servicemesh-operator3-55f49c5f94-dnk6f\" (UID: \"08ec4678-017b-451e-93d4-c570b48b5e75\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f" May 20 19:08:24.065467 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:24.065427 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f" May 20 19:08:24.257236 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:24.257190 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f"] May 20 19:08:24.260779 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:08:24.260710 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08ec4678_017b_451e_93d4_c570b48b5e75.slice/crio-9489a76ee09c8274ec62db77997b8969c88d541d58c441941544853eeec64407 WatchSource:0}: Error finding container 9489a76ee09c8274ec62db77997b8969c88d541d58c441941544853eeec64407: Status 404 returned error can't find the container with id 9489a76ee09c8274ec62db77997b8969c88d541d58c441941544853eeec64407 May 20 19:08:24.635060 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:24.635022 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f" event={"ID":"08ec4678-017b-451e-93d4-c570b48b5e75","Type":"ContainerStarted","Data":"9489a76ee09c8274ec62db77997b8969c88d541d58c441941544853eeec64407"} May 20 19:08:27.028661 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:27.028568 2575 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:27.029110 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:27.029093 2575 scope.go:117] "RemoveContainer" containerID="f5e050be3f36dddf94b47ac81571fb850778197a63d9325915cfdc57c6a25154" May 20 19:08:27.646420 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:27.646325 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" event={"ID":"13823e31-0602-45a5-bb05-e980ef446c1c","Type":"ContainerStarted","Data":"eeff8018a4b3bb5c21776f3638901f042c322badf3b4489a966502909f9bec5f"} May 20 19:08:27.646613 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:27.646593 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:27.647860 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:27.647826 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f" event={"ID":"08ec4678-017b-451e-93d4-c570b48b5e75","Type":"ContainerStarted","Data":"af932398e535a671731b5bada95f966beca826482492bb74522e11afc89c745b"} May 20 19:08:27.647979 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:27.647953 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f" May 20 19:08:27.699589 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:27.699538 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" podStartSLOduration=2.562194333 podStartE2EDuration="22.699524265s" podCreationTimestamp="2026-05-20 19:08:05 +0000 UTC" firstStartedPulling="2026-05-20 19:08:07.170762588 +0000 UTC m=+544.716721437" lastFinishedPulling="2026-05-20 19:08:27.308092522 +0000 UTC m=+564.854051369" observedRunningTime="2026-05-20 19:08:27.697547455 +0000 UTC m=+565.243506309" watchObservedRunningTime="2026-05-20 19:08:27.699524265 +0000 UTC m=+565.245483121" May 20 19:08:27.734702 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:27.734647 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f" podStartSLOduration=2.363988584 podStartE2EDuration="4.734629625s" podCreationTimestamp="2026-05-20 19:08:23 +0000 UTC" firstStartedPulling="2026-05-20 19:08:24.263200729 +0000 UTC m=+561.809159561" lastFinishedPulling="2026-05-20 19:08:26.63384177 +0000 UTC m=+564.179800602" observedRunningTime="2026-05-20 19:08:27.732322649 +0000 UTC m=+565.278281506" watchObservedRunningTime="2026-05-20 19:08:27.734629625 +0000 UTC m=+565.280588479" May 20 19:08:33.221136 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.221103 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b"] May 20 19:08:33.224051 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.224034 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.229336 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.229314 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" May 20 19:08:33.229454 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.229317 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" May 20 19:08:33.230554 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.230313 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" May 20 19:08:33.230554 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.230326 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" May 20 19:08:33.230554 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.230347 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dr2pf\"" May 20 19:08:33.251522 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.251478 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b"] May 20 19:08:33.392678 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.392634 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.392920 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.392879 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-cacerts\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.393071 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.392930 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.393071 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.392952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.393179 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.393104 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-istio-token\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.393179 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.393146 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-local-certs\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.393306 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.393190 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4j6p\" (UniqueName: \"kubernetes.io/projected/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-kube-api-access-g4j6p\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.493812 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.493722 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-cacerts\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.493812 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.493757 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.493812 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.493775 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.494098 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.493829 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-istio-token\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.494098 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.493859 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-local-certs\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.494098 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.493882 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g4j6p\" (UniqueName: \"kubernetes.io/projected/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-kube-api-access-g4j6p\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.494098 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.493922 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.494481 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.494450 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.496743 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.496710 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-local-certs\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.497318 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.497287 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-cacerts\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.497505 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.497482 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.497630 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.497610 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.519461 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.519426 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-istio-token\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.520637 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.520610 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4j6p\" (UniqueName: \"kubernetes.io/projected/fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0-kube-api-access-g4j6p\") pod \"istiod-openshift-gateway-798958bb55-gcf2b\" (UID: \"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.533443 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.533406 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:33.717578 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:33.717541 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b"] May 20 19:08:33.717710 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:08:33.717644 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb1fea04_5a17_4fc8_9ae7_79d5a27f47c0.slice/crio-e4b0b62499a71a5d715a151157f020fef5221309467c109ea4e517a873e3d74e WatchSource:0}: Error finding container e4b0b62499a71a5d715a151157f020fef5221309467c109ea4e517a873e3d74e: Status 404 returned error can't find the container with id e4b0b62499a71a5d715a151157f020fef5221309467c109ea4e517a873e3d74e May 20 19:08:34.671778 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:34.671739 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" event={"ID":"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0","Type":"ContainerStarted","Data":"e4b0b62499a71a5d715a151157f020fef5221309467c109ea4e517a873e3d74e"} May 20 19:08:36.367739 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:36.367698 2575 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} May 20 19:08:36.368129 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:36.367768 2575 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} May 20 19:08:36.680982 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:36.680891 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" event={"ID":"fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0","Type":"ContainerStarted","Data":"8f0f2caa67c76a8bd9e1ee7d7b34a256e17e8136d4d184387f69704fdc4cc78a"} May 20 19:08:36.681164 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:36.681004 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:36.710603 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:36.710544 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" podStartSLOduration=1.062806053 podStartE2EDuration="3.71052099s" podCreationTimestamp="2026-05-20 19:08:33 +0000 UTC" firstStartedPulling="2026-05-20 19:08:33.719728322 +0000 UTC m=+571.265687155" lastFinishedPulling="2026-05-20 19:08:36.367443259 +0000 UTC m=+573.913402092" observedRunningTime="2026-05-20 19:08:36.707426023 +0000 UTC m=+574.253384879" watchObservedRunningTime="2026-05-20 19:08:36.71052099 +0000 UTC m=+574.256479846" May 20 19:08:37.686302 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:37.686263 2575 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-gcf2b container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= May 20 19:08:37.686760 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:37.686343 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" podUID="fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 20 19:08:38.653155 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:38.653128 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-dnk6f" May 20 19:08:38.653372 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:38.653184 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-qlgtq" May 20 19:08:40.685888 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:40.685849 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-gcf2b" May 20 19:08:46.618315 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:08:46.618281 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-6lwqb" May 20 19:09:02.993508 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:02.993470 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:09:03.001039 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:03.000999 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:09:40.106347 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.106315 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk"] May 20 19:09:40.108416 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.108397 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" May 20 19:09:40.116355 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.116333 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 20 19:09:40.116442 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.116353 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sdqjb\"" May 20 19:09:40.116442 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.116337 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 20 19:09:40.128624 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.128595 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk"] May 20 19:09:40.155530 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.155491 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13b7b181-6c61-4cf4-83f8-c262e7e8a6d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-x9ssk\" (UID: \"13b7b181-6c61-4cf4-83f8-c262e7e8a6d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" May 20 19:09:40.155693 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.155560 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4zsr\" (UniqueName: \"kubernetes.io/projected/13b7b181-6c61-4cf4-83f8-c262e7e8a6d9-kube-api-access-g4zsr\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-x9ssk\" (UID: \"13b7b181-6c61-4cf4-83f8-c262e7e8a6d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" May 20 19:09:40.256103 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.256018 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g4zsr\" (UniqueName: \"kubernetes.io/projected/13b7b181-6c61-4cf4-83f8-c262e7e8a6d9-kube-api-access-g4zsr\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-x9ssk\" (UID: \"13b7b181-6c61-4cf4-83f8-c262e7e8a6d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" May 20 19:09:40.256103 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.256083 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13b7b181-6c61-4cf4-83f8-c262e7e8a6d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-x9ssk\" (UID: \"13b7b181-6c61-4cf4-83f8-c262e7e8a6d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" May 20 19:09:40.256435 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.256418 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13b7b181-6c61-4cf4-83f8-c262e7e8a6d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-x9ssk\" (UID: \"13b7b181-6c61-4cf4-83f8-c262e7e8a6d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" May 20 19:09:40.268376 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.268346 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4zsr\" (UniqueName: \"kubernetes.io/projected/13b7b181-6c61-4cf4-83f8-c262e7e8a6d9-kube-api-access-g4zsr\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-x9ssk\" (UID: \"13b7b181-6c61-4cf4-83f8-c262e7e8a6d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" May 20 19:09:40.419167 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.419131 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" May 20 19:09:40.560898 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.560873 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk"] May 20 19:09:40.563439 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:09:40.563390 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b7b181_6c61_4cf4_83f8_c262e7e8a6d9.slice/crio-333db2f411d71cb90ea1f900f2089b90b6518fb21f040e88a92c7b221f7e970d WatchSource:0}: Error finding container 333db2f411d71cb90ea1f900f2089b90b6518fb21f040e88a92c7b221f7e970d: Status 404 returned error can't find the container with id 333db2f411d71cb90ea1f900f2089b90b6518fb21f040e88a92c7b221f7e970d May 20 19:09:40.878218 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:40.878123 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" event={"ID":"13b7b181-6c61-4cf4-83f8-c262e7e8a6d9","Type":"ContainerStarted","Data":"333db2f411d71cb90ea1f900f2089b90b6518fb21f040e88a92c7b221f7e970d"} May 20 19:09:46.386570 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.386536 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-bc9kf"] May 20 19:09:46.389188 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.389170 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bc9kf" May 20 19:09:46.391971 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.391949 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-kcfrm\"" May 20 19:09:46.392279 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.392265 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" May 20 19:09:46.409322 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.409294 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqlbp\" (UniqueName: \"kubernetes.io/projected/92c15793-b2d1-402e-bb21-766b22dd60dd-kube-api-access-dqlbp\") pod \"dns-operator-controller-manager-648d5c98bc-bc9kf\" (UID: \"92c15793-b2d1-402e-bb21-766b22dd60dd\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bc9kf" May 20 19:09:46.410926 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.410903 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-bc9kf"] May 20 19:09:46.510641 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.510604 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dqlbp\" (UniqueName: \"kubernetes.io/projected/92c15793-b2d1-402e-bb21-766b22dd60dd-kube-api-access-dqlbp\") pod \"dns-operator-controller-manager-648d5c98bc-bc9kf\" (UID: \"92c15793-b2d1-402e-bb21-766b22dd60dd\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bc9kf" May 20 19:09:46.521819 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.521794 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqlbp\" (UniqueName: \"kubernetes.io/projected/92c15793-b2d1-402e-bb21-766b22dd60dd-kube-api-access-dqlbp\") pod \"dns-operator-controller-manager-648d5c98bc-bc9kf\" (UID: \"92c15793-b2d1-402e-bb21-766b22dd60dd\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bc9kf" May 20 19:09:46.699049 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.698976 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bc9kf" May 20 19:09:46.844396 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:09:46.844367 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c15793_b2d1_402e_bb21_766b22dd60dd.slice/crio-3681819437c9c76b253db4bd072aa91fd0c011aa6b9adb2a2862c880a85fa2af WatchSource:0}: Error finding container 3681819437c9c76b253db4bd072aa91fd0c011aa6b9adb2a2862c880a85fa2af: Status 404 returned error can't find the container with id 3681819437c9c76b253db4bd072aa91fd0c011aa6b9adb2a2862c880a85fa2af May 20 19:09:46.844729 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.844710 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-bc9kf"] May 20 19:09:46.900691 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.900660 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bc9kf" event={"ID":"92c15793-b2d1-402e-bb21-766b22dd60dd","Type":"ContainerStarted","Data":"3681819437c9c76b253db4bd072aa91fd0c011aa6b9adb2a2862c880a85fa2af"} May 20 19:09:46.901946 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.901924 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" event={"ID":"13b7b181-6c61-4cf4-83f8-c262e7e8a6d9","Type":"ContainerStarted","Data":"d55ae6c49d5f1b7a2dc9f927839b46e4f2dce4848abf6e87bec3359c00fc8247"} May 20 19:09:46.902099 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.902085 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" May 20 19:09:46.943563 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:46.943518 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" podStartSLOduration=1.205374271 podStartE2EDuration="6.943503755s" podCreationTimestamp="2026-05-20 19:09:40 +0000 UTC" firstStartedPulling="2026-05-20 19:09:40.565691853 +0000 UTC m=+638.111650688" lastFinishedPulling="2026-05-20 19:09:46.303821335 +0000 UTC m=+643.849780172" observedRunningTime="2026-05-20 19:09:46.940363986 +0000 UTC m=+644.486322900" watchObservedRunningTime="2026-05-20 19:09:46.943503755 +0000 UTC m=+644.489462677" May 20 19:09:49.914997 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:49.914957 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bc9kf" event={"ID":"92c15793-b2d1-402e-bb21-766b22dd60dd","Type":"ContainerStarted","Data":"81219afc8bf451188bd0e4b91534a39f45c8baabd0346a0746e6497255e32054"} May 20 19:09:49.915393 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:49.915087 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bc9kf" May 20 19:09:49.949898 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:49.949855 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bc9kf" podStartSLOduration=1.82886732 podStartE2EDuration="3.949840587s" podCreationTimestamp="2026-05-20 19:09:46 +0000 UTC" firstStartedPulling="2026-05-20 19:09:46.846103569 +0000 UTC m=+644.392062402" lastFinishedPulling="2026-05-20 19:09:48.96707682 +0000 UTC m=+646.513035669" observedRunningTime="2026-05-20 19:09:49.949499449 +0000 UTC m=+647.495458324" watchObservedRunningTime="2026-05-20 19:09:49.949840587 +0000 UTC m=+647.495799442" May 20 19:09:57.908366 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:57.908331 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" May 20 19:09:59.738423 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.738393 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk"] May 20 19:09:59.738750 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.738606 2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" podUID="13b7b181-6c61-4cf4-83f8-c262e7e8a6d9" containerName="manager" containerID="cri-o://d55ae6c49d5f1b7a2dc9f927839b46e4f2dce4848abf6e87bec3359c00fc8247" gracePeriod=2 May 20 19:09:59.754197 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.754173 2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk"] May 20 19:09:59.779042 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.779017 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95"] May 20 19:09:59.779443 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.779425 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b7b181-6c61-4cf4-83f8-c262e7e8a6d9" containerName="manager" May 20 19:09:59.779532 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.779447 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b7b181-6c61-4cf4-83f8-c262e7e8a6d9" containerName="manager" May 20 19:09:59.779532 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.779524 2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="13b7b181-6c61-4cf4-83f8-c262e7e8a6d9" containerName="manager" May 20 19:09:59.781498 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.781480 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95" May 20 19:09:59.801340 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.801311 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95"] May 20 19:09:59.828185 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.828153 2575 status_manager.go:895] "Failed to get status for pod" podUID="13b7b181-6c61-4cf4-83f8-c262e7e8a6d9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-x9ssk\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" May 20 19:09:59.829502 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.829482 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/229c90f0-7516-4df6-903c-74f1379aac0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-jfc95\" (UID: \"229c90f0-7516-4df6-903c-74f1379aac0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95" May 20 19:09:59.829576 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.829563 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhcbm\" (UniqueName: \"kubernetes.io/projected/229c90f0-7516-4df6-903c-74f1379aac0f-kube-api-access-lhcbm\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-jfc95\" (UID: \"229c90f0-7516-4df6-903c-74f1379aac0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95" May 20 19:09:59.930991 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.930960 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lhcbm\" (UniqueName: \"kubernetes.io/projected/229c90f0-7516-4df6-903c-74f1379aac0f-kube-api-access-lhcbm\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-jfc95\" (UID: \"229c90f0-7516-4df6-903c-74f1379aac0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95" May 20 19:09:59.931132 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.931014 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/229c90f0-7516-4df6-903c-74f1379aac0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-jfc95\" (UID: \"229c90f0-7516-4df6-903c-74f1379aac0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95" May 20 19:09:59.931357 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.931340 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/229c90f0-7516-4df6-903c-74f1379aac0f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-jfc95\" (UID: \"229c90f0-7516-4df6-903c-74f1379aac0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95" May 20 19:09:59.949532 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.949492 2575 generic.go:358] "Generic (PLEG): container finished" podID="13b7b181-6c61-4cf4-83f8-c262e7e8a6d9" containerID="d55ae6c49d5f1b7a2dc9f927839b46e4f2dce4848abf6e87bec3359c00fc8247" exitCode=0 May 20 19:09:59.957427 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.957403 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95"] May 20 19:09:59.957706 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:09:59.957683 2575 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-lhcbm], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95" podUID="229c90f0-7516-4df6-903c-74f1379aac0f" May 20 19:09:59.958805 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.958781 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhcbm\" (UniqueName: \"kubernetes.io/projected/229c90f0-7516-4df6-903c-74f1379aac0f-kube-api-access-lhcbm\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-jfc95\" (UID: \"229c90f0-7516-4df6-903c-74f1379aac0f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95" May 20 19:09:59.975760 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.975742 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" May 20 19:09:59.978677 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:09:59.978649 2575 status_manager.go:895] "Failed to get status for pod" podUID="13b7b181-6c61-4cf4-83f8-c262e7e8a6d9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-x9ssk\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" May 20 19:10:00.005082 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.005012 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8"] May 20 19:10:00.007266 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.007250 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" May 20 19:10:00.009521 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.009497 2575 status_manager.go:895] "Failed to get status for pod" podUID="13b7b181-6c61-4cf4-83f8-c262e7e8a6d9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-x9ssk\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" May 20 19:10:00.026571 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.026547 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8"] May 20 19:10:00.032148 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.032126 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4zsr\" (UniqueName: \"kubernetes.io/projected/13b7b181-6c61-4cf4-83f8-c262e7e8a6d9-kube-api-access-g4zsr\") pod \"13b7b181-6c61-4cf4-83f8-c262e7e8a6d9\" (UID: \"13b7b181-6c61-4cf4-83f8-c262e7e8a6d9\") " May 20 19:10:00.032274 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.032172 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13b7b181-6c61-4cf4-83f8-c262e7e8a6d9-extensions-socket-volume\") pod \"13b7b181-6c61-4cf4-83f8-c262e7e8a6d9\" (UID: \"13b7b181-6c61-4cf4-83f8-c262e7e8a6d9\") " May 20 19:10:00.032617 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.032595 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b7b181-6c61-4cf4-83f8-c262e7e8a6d9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "13b7b181-6c61-4cf4-83f8-c262e7e8a6d9" (UID: "13b7b181-6c61-4cf4-83f8-c262e7e8a6d9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 20 19:10:00.034411 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.034388 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b7b181-6c61-4cf4-83f8-c262e7e8a6d9-kube-api-access-g4zsr" (OuterVolumeSpecName: "kube-api-access-g4zsr") pod "13b7b181-6c61-4cf4-83f8-c262e7e8a6d9" (UID: "13b7b181-6c61-4cf4-83f8-c262e7e8a6d9"). InnerVolumeSpecName "kube-api-access-g4zsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 19:10:00.133264 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.133227 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b53d672b-81cb-4d66-bc86-afaa95414437-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-pxtn8\" (UID: \"b53d672b-81cb-4d66-bc86-afaa95414437\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" May 20 19:10:00.133430 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.133338 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqdzp\" (UniqueName: \"kubernetes.io/projected/b53d672b-81cb-4d66-bc86-afaa95414437-kube-api-access-nqdzp\") pod \"kuadrant-operator-controller-manager-55c7f4c975-pxtn8\" (UID: \"b53d672b-81cb-4d66-bc86-afaa95414437\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" May 20 19:10:00.133430 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.133406 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4zsr\" (UniqueName: \"kubernetes.io/projected/13b7b181-6c61-4cf4-83f8-c262e7e8a6d9-kube-api-access-g4zsr\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:10:00.133430 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.133419 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13b7b181-6c61-4cf4-83f8-c262e7e8a6d9-extensions-socket-volume\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:10:00.234348 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.234312 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b53d672b-81cb-4d66-bc86-afaa95414437-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-pxtn8\" (UID: \"b53d672b-81cb-4d66-bc86-afaa95414437\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" May 20 19:10:00.234504 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.234387 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nqdzp\" (UniqueName: \"kubernetes.io/projected/b53d672b-81cb-4d66-bc86-afaa95414437-kube-api-access-nqdzp\") pod \"kuadrant-operator-controller-manager-55c7f4c975-pxtn8\" (UID: \"b53d672b-81cb-4d66-bc86-afaa95414437\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" May 20 19:10:00.234699 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.234681 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b53d672b-81cb-4d66-bc86-afaa95414437-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-pxtn8\" (UID: \"b53d672b-81cb-4d66-bc86-afaa95414437\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" May 20 19:10:00.252315 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.252286 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqdzp\" (UniqueName: \"kubernetes.io/projected/b53d672b-81cb-4d66-bc86-afaa95414437-kube-api-access-nqdzp\") pod \"kuadrant-operator-controller-manager-55c7f4c975-pxtn8\" (UID: \"b53d672b-81cb-4d66-bc86-afaa95414437\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" May 20 19:10:00.317258 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.317225 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" May 20 19:10:00.455663 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.455637 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8"] May 20 19:10:00.457738 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:10:00.457705 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb53d672b_81cb_4d66_bc86_afaa95414437.slice/crio-8b1b87537dbb6ac4c8d0d2f39db55ed8a9e1376ba2536759450ac11b40fa7e0d WatchSource:0}: Error finding container 8b1b87537dbb6ac4c8d0d2f39db55ed8a9e1376ba2536759450ac11b40fa7e0d: Status 404 returned error can't find the container with id 8b1b87537dbb6ac4c8d0d2f39db55ed8a9e1376ba2536759450ac11b40fa7e0d May 20 19:10:00.920619 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.920586 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bc9kf" May 20 19:10:00.924128 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.924102 2575 status_manager.go:895] "Failed to get status for pod" podUID="13b7b181-6c61-4cf4-83f8-c262e7e8a6d9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-x9ssk\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" May 20 19:10:00.954826 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.954793 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" event={"ID":"b53d672b-81cb-4d66-bc86-afaa95414437","Type":"ContainerStarted","Data":"32ce2d6756186d3a09d94574491fe40fce09e17943a8bf0cb6f4745a531759c7"} May 20 19:10:00.954980 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.954833 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" event={"ID":"b53d672b-81cb-4d66-bc86-afaa95414437","Type":"ContainerStarted","Data":"8b1b87537dbb6ac4c8d0d2f39db55ed8a9e1376ba2536759450ac11b40fa7e0d"} May 20 19:10:00.954980 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.954877 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" May 20 19:10:00.956098 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.956081 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" May 20 19:10:00.956098 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.956098 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95" May 20 19:10:00.956267 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.956138 2575 scope.go:117] "RemoveContainer" containerID="d55ae6c49d5f1b7a2dc9f927839b46e4f2dce4848abf6e87bec3359c00fc8247" May 20 19:10:00.962810 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:00.962790 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95" May 20 19:10:01.020106 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:01.020072 2575 status_manager.go:895] "Failed to get status for pod" podUID="13b7b181-6c61-4cf4-83f8-c262e7e8a6d9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-x9ssk" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-x9ssk\" is forbidden: User \"system:node:ip-10-0-138-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-66.ec2.internal' and this object" May 20 19:10:01.041847 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:01.041817 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/229c90f0-7516-4df6-903c-74f1379aac0f-extensions-socket-volume\") pod \"229c90f0-7516-4df6-903c-74f1379aac0f\" (UID: \"229c90f0-7516-4df6-903c-74f1379aac0f\") " May 20 19:10:01.041988 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:01.041862 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhcbm\" (UniqueName: \"kubernetes.io/projected/229c90f0-7516-4df6-903c-74f1379aac0f-kube-api-access-lhcbm\") pod \"229c90f0-7516-4df6-903c-74f1379aac0f\" (UID: \"229c90f0-7516-4df6-903c-74f1379aac0f\") " May 20 19:10:01.042073 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:01.042047 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/229c90f0-7516-4df6-903c-74f1379aac0f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "229c90f0-7516-4df6-903c-74f1379aac0f" (UID: "229c90f0-7516-4df6-903c-74f1379aac0f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 20 19:10:01.042184 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:01.042165 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/229c90f0-7516-4df6-903c-74f1379aac0f-extensions-socket-volume\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:10:01.044150 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:01.044129 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/229c90f0-7516-4df6-903c-74f1379aac0f-kube-api-access-lhcbm" (OuterVolumeSpecName: "kube-api-access-lhcbm") pod "229c90f0-7516-4df6-903c-74f1379aac0f" (UID: "229c90f0-7516-4df6-903c-74f1379aac0f"). InnerVolumeSpecName "kube-api-access-lhcbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 19:10:01.066680 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:01.066633 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" podStartSLOduration=2.066616705 podStartE2EDuration="2.066616705s" podCreationTimestamp="2026-05-20 19:09:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 19:10:01.065957998 +0000 UTC m=+658.611916869" watchObservedRunningTime="2026-05-20 19:10:01.066616705 +0000 UTC m=+658.612575559" May 20 19:10:01.089269 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:01.089244 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13b7b181-6c61-4cf4-83f8-c262e7e8a6d9" path="/var/lib/kubelet/pods/13b7b181-6c61-4cf4-83f8-c262e7e8a6d9/volumes" May 20 19:10:01.143239 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:01.143191 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhcbm\" (UniqueName: \"kubernetes.io/projected/229c90f0-7516-4df6-903c-74f1379aac0f-kube-api-access-lhcbm\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:10:01.960565 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:01.960538 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95" May 20 19:10:02.019072 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:02.019037 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95"] May 20 19:10:02.024120 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:02.024093 2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-jfc95"] May 20 19:10:03.089762 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:03.089725 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="229c90f0-7516-4df6-903c-74f1379aac0f" path="/var/lib/kubelet/pods/229c90f0-7516-4df6-903c-74f1379aac0f/volumes" May 20 19:10:11.963197 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:11.963165 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" May 20 19:10:32.624635 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:32.624591 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-bjkz4"] May 20 19:10:32.627549 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:32.627523 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" May 20 19:10:32.630142 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:32.630123 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xfz9v\"" May 20 19:10:32.630320 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:32.630300 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" May 20 19:10:32.638264 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:32.638240 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-bjkz4"] May 20 19:10:32.711725 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:32.711694 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t6bl\" (UniqueName: \"kubernetes.io/projected/f97defd9-7166-475c-b2e4-5f22345bd3b7-kube-api-access-9t6bl\") pod \"limitador-limitador-7d549b5b-bjkz4\" (UID: \"f97defd9-7166-475c-b2e4-5f22345bd3b7\") " pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" May 20 19:10:32.711895 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:32.711754 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f97defd9-7166-475c-b2e4-5f22345bd3b7-config-file\") pod \"limitador-limitador-7d549b5b-bjkz4\" (UID: \"f97defd9-7166-475c-b2e4-5f22345bd3b7\") " pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" May 20 19:10:32.723686 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:32.723653 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-bjkz4"] May 20 19:10:32.812564 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:32.812529 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9t6bl\" (UniqueName: \"kubernetes.io/projected/f97defd9-7166-475c-b2e4-5f22345bd3b7-kube-api-access-9t6bl\") pod \"limitador-limitador-7d549b5b-bjkz4\" (UID: \"f97defd9-7166-475c-b2e4-5f22345bd3b7\") " pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" May 20 19:10:32.812768 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:32.812593 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f97defd9-7166-475c-b2e4-5f22345bd3b7-config-file\") pod \"limitador-limitador-7d549b5b-bjkz4\" (UID: \"f97defd9-7166-475c-b2e4-5f22345bd3b7\") " pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" May 20 19:10:32.813172 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:32.813152 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f97defd9-7166-475c-b2e4-5f22345bd3b7-config-file\") pod \"limitador-limitador-7d549b5b-bjkz4\" (UID: \"f97defd9-7166-475c-b2e4-5f22345bd3b7\") " pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" May 20 19:10:32.822681 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:32.822654 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t6bl\" (UniqueName: \"kubernetes.io/projected/f97defd9-7166-475c-b2e4-5f22345bd3b7-kube-api-access-9t6bl\") pod \"limitador-limitador-7d549b5b-bjkz4\" (UID: \"f97defd9-7166-475c-b2e4-5f22345bd3b7\") " pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" May 20 19:10:32.938424 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:32.938342 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" May 20 19:10:33.075266 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:33.075084 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-bjkz4"] May 20 19:10:33.077831 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:10:33.077805 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf97defd9_7166_475c_b2e4_5f22345bd3b7.slice/crio-23cd70c9f3c3804d6a5e82fd7c7eead4880e6c0adc41c915a51dfea972f48467 WatchSource:0}: Error finding container 23cd70c9f3c3804d6a5e82fd7c7eead4880e6c0adc41c915a51dfea972f48467: Status 404 returned error can't find the container with id 23cd70c9f3c3804d6a5e82fd7c7eead4880e6c0adc41c915a51dfea972f48467 May 20 19:10:34.065752 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:34.065709 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" event={"ID":"f97defd9-7166-475c-b2e4-5f22345bd3b7","Type":"ContainerStarted","Data":"23cd70c9f3c3804d6a5e82fd7c7eead4880e6c0adc41c915a51dfea972f48467"} May 20 19:10:36.073828 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:36.073786 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" event={"ID":"f97defd9-7166-475c-b2e4-5f22345bd3b7","Type":"ContainerStarted","Data":"f442c41ac290d8cf7c838564d868660117e2fab45b99d1bd6af5dbd9031ec8da"} May 20 19:10:36.074172 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:36.073935 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" May 20 19:10:36.092561 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:36.092474 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" podStartSLOduration=1.335966631 podStartE2EDuration="4.092461489s" podCreationTimestamp="2026-05-20 19:10:32 +0000 UTC" firstStartedPulling="2026-05-20 19:10:33.079430727 +0000 UTC m=+690.625389559" lastFinishedPulling="2026-05-20 19:10:35.835925574 +0000 UTC m=+693.381884417" observedRunningTime="2026-05-20 19:10:36.090907361 +0000 UTC m=+693.636866215" watchObservedRunningTime="2026-05-20 19:10:36.092461489 +0000 UTC m=+693.638420343" May 20 19:10:47.081772 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:47.081739 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" May 20 19:10:49.257657 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:49.257621 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-bjkz4"] May 20 19:10:49.258110 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:49.257910 2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" podUID="f97defd9-7166-475c-b2e4-5f22345bd3b7" containerName="limitador" containerID="cri-o://f442c41ac290d8cf7c838564d868660117e2fab45b99d1bd6af5dbd9031ec8da" gracePeriod=30 May 20 19:10:49.800050 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:49.800023 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" May 20 19:10:49.828084 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:49.828015 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t6bl\" (UniqueName: \"kubernetes.io/projected/f97defd9-7166-475c-b2e4-5f22345bd3b7-kube-api-access-9t6bl\") pod \"f97defd9-7166-475c-b2e4-5f22345bd3b7\" (UID: \"f97defd9-7166-475c-b2e4-5f22345bd3b7\") " May 20 19:10:49.828231 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:49.828119 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f97defd9-7166-475c-b2e4-5f22345bd3b7-config-file\") pod \"f97defd9-7166-475c-b2e4-5f22345bd3b7\" (UID: \"f97defd9-7166-475c-b2e4-5f22345bd3b7\") " May 20 19:10:49.828555 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:49.828531 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f97defd9-7166-475c-b2e4-5f22345bd3b7-config-file" (OuterVolumeSpecName: "config-file") pod "f97defd9-7166-475c-b2e4-5f22345bd3b7" (UID: "f97defd9-7166-475c-b2e4-5f22345bd3b7"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 20 19:10:49.830042 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:49.830022 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f97defd9-7166-475c-b2e4-5f22345bd3b7-kube-api-access-9t6bl" (OuterVolumeSpecName: "kube-api-access-9t6bl") pod "f97defd9-7166-475c-b2e4-5f22345bd3b7" (UID: "f97defd9-7166-475c-b2e4-5f22345bd3b7"). InnerVolumeSpecName "kube-api-access-9t6bl". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 19:10:49.928689 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:49.928656 2575 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f97defd9-7166-475c-b2e4-5f22345bd3b7-config-file\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:10:49.928689 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:49.928687 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t6bl\" (UniqueName: \"kubernetes.io/projected/f97defd9-7166-475c-b2e4-5f22345bd3b7-kube-api-access-9t6bl\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:10:50.118992 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:50.118883 2575 generic.go:358] "Generic (PLEG): container finished" podID="f97defd9-7166-475c-b2e4-5f22345bd3b7" containerID="f442c41ac290d8cf7c838564d868660117e2fab45b99d1bd6af5dbd9031ec8da" exitCode=0 May 20 19:10:50.118992 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:50.118921 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" event={"ID":"f97defd9-7166-475c-b2e4-5f22345bd3b7","Type":"ContainerDied","Data":"f442c41ac290d8cf7c838564d868660117e2fab45b99d1bd6af5dbd9031ec8da"} May 20 19:10:50.118992 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:50.118943 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" event={"ID":"f97defd9-7166-475c-b2e4-5f22345bd3b7","Type":"ContainerDied","Data":"23cd70c9f3c3804d6a5e82fd7c7eead4880e6c0adc41c915a51dfea972f48467"} May 20 19:10:50.118992 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:50.118949 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-bjkz4" May 20 19:10:50.118992 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:50.118958 2575 scope.go:117] "RemoveContainer" containerID="f442c41ac290d8cf7c838564d868660117e2fab45b99d1bd6af5dbd9031ec8da" May 20 19:10:50.127166 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:50.127144 2575 scope.go:117] "RemoveContainer" containerID="f442c41ac290d8cf7c838564d868660117e2fab45b99d1bd6af5dbd9031ec8da" May 20 19:10:50.127443 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:10:50.127423 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f442c41ac290d8cf7c838564d868660117e2fab45b99d1bd6af5dbd9031ec8da\": container with ID starting with f442c41ac290d8cf7c838564d868660117e2fab45b99d1bd6af5dbd9031ec8da not found: ID does not exist" containerID="f442c41ac290d8cf7c838564d868660117e2fab45b99d1bd6af5dbd9031ec8da" May 20 19:10:50.127502 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:50.127452 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f442c41ac290d8cf7c838564d868660117e2fab45b99d1bd6af5dbd9031ec8da"} err="failed to get container status \"f442c41ac290d8cf7c838564d868660117e2fab45b99d1bd6af5dbd9031ec8da\": rpc error: code = NotFound desc = could not find container \"f442c41ac290d8cf7c838564d868660117e2fab45b99d1bd6af5dbd9031ec8da\": container with ID starting with f442c41ac290d8cf7c838564d868660117e2fab45b99d1bd6af5dbd9031ec8da not found: ID does not exist" May 20 19:10:50.144023 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:50.143996 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-bjkz4"] May 20 19:10:50.150505 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:50.150480 2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-bjkz4"] May 20 19:10:51.089672 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:51.089635 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f97defd9-7166-475c-b2e4-5f22345bd3b7" path="/var/lib/kubelet/pods/f97defd9-7166-475c-b2e4-5f22345bd3b7/volumes" May 20 19:10:53.928241 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:53.928185 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-kx5fw"] May 20 19:10:53.934098 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:53.928885 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f97defd9-7166-475c-b2e4-5f22345bd3b7" containerName="limitador" May 20 19:10:53.934098 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:53.928911 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f97defd9-7166-475c-b2e4-5f22345bd3b7" containerName="limitador" May 20 19:10:53.934098 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:53.929091 2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="f97defd9-7166-475c-b2e4-5f22345bd3b7" containerName="limitador" May 20 19:10:53.934098 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:53.932831 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-kx5fw" May 20 19:10:53.936402 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:53.936365 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" May 20 19:10:53.937370 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:53.937340 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tkftb\"" May 20 19:10:53.940298 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:53.940269 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-kx5fw"] May 20 19:10:53.957442 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:53.957411 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d999v\" (UniqueName: \"kubernetes.io/projected/7430e17b-f4a9-44c4-8922-9e4fd7510d94-kube-api-access-d999v\") pod \"postgres-68c4fbbd6f-kx5fw\" (UID: \"7430e17b-f4a9-44c4-8922-9e4fd7510d94\") " pod="opendatahub/postgres-68c4fbbd6f-kx5fw" May 20 19:10:53.957615 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:53.957525 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7430e17b-f4a9-44c4-8922-9e4fd7510d94-data\") pod \"postgres-68c4fbbd6f-kx5fw\" (UID: \"7430e17b-f4a9-44c4-8922-9e4fd7510d94\") " pod="opendatahub/postgres-68c4fbbd6f-kx5fw" May 20 19:10:54.058348 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:54.058308 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d999v\" (UniqueName: \"kubernetes.io/projected/7430e17b-f4a9-44c4-8922-9e4fd7510d94-kube-api-access-d999v\") pod \"postgres-68c4fbbd6f-kx5fw\" (UID: \"7430e17b-f4a9-44c4-8922-9e4fd7510d94\") " pod="opendatahub/postgres-68c4fbbd6f-kx5fw" May 20 19:10:54.058565 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:54.058391 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7430e17b-f4a9-44c4-8922-9e4fd7510d94-data\") pod \"postgres-68c4fbbd6f-kx5fw\" (UID: \"7430e17b-f4a9-44c4-8922-9e4fd7510d94\") " pod="opendatahub/postgres-68c4fbbd6f-kx5fw" May 20 19:10:54.058761 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:54.058739 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7430e17b-f4a9-44c4-8922-9e4fd7510d94-data\") pod \"postgres-68c4fbbd6f-kx5fw\" (UID: \"7430e17b-f4a9-44c4-8922-9e4fd7510d94\") " pod="opendatahub/postgres-68c4fbbd6f-kx5fw" May 20 19:10:54.068568 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:54.068533 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d999v\" (UniqueName: \"kubernetes.io/projected/7430e17b-f4a9-44c4-8922-9e4fd7510d94-kube-api-access-d999v\") pod \"postgres-68c4fbbd6f-kx5fw\" (UID: \"7430e17b-f4a9-44c4-8922-9e4fd7510d94\") " pod="opendatahub/postgres-68c4fbbd6f-kx5fw" May 20 19:10:54.247199 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:54.247112 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-kx5fw" May 20 19:10:54.375049 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:54.375017 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-kx5fw"] May 20 19:10:54.377455 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:10:54.377427 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7430e17b_f4a9_44c4_8922_9e4fd7510d94.slice/crio-0a6a1561963542ec3132a65cbeb5e0f909ef5cd0c467d547df9db10b09c616e0 WatchSource:0}: Error finding container 0a6a1561963542ec3132a65cbeb5e0f909ef5cd0c467d547df9db10b09c616e0: Status 404 returned error can't find the container with id 0a6a1561963542ec3132a65cbeb5e0f909ef5cd0c467d547df9db10b09c616e0 May 20 19:10:55.135793 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:10:55.135754 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-kx5fw" event={"ID":"7430e17b-f4a9-44c4-8922-9e4fd7510d94","Type":"ContainerStarted","Data":"0a6a1561963542ec3132a65cbeb5e0f909ef5cd0c467d547df9db10b09c616e0"} May 20 19:11:00.155489 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:00.155449 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-kx5fw" event={"ID":"7430e17b-f4a9-44c4-8922-9e4fd7510d94","Type":"ContainerStarted","Data":"f28320456d369dad4a0f944ff3889a3faffbde2dab09e386262e34368b49cf05"} May 20 19:11:00.155926 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:00.155568 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-kx5fw" May 20 19:11:00.179474 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:00.179424 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-kx5fw" podStartSLOduration=1.970751048 podStartE2EDuration="7.179407457s" podCreationTimestamp="2026-05-20 19:10:53 +0000 UTC" firstStartedPulling="2026-05-20 19:10:54.378599852 +0000 UTC m=+711.924558685" lastFinishedPulling="2026-05-20 19:10:59.587256247 +0000 UTC m=+717.133215094" observedRunningTime="2026-05-20 19:11:00.178940716 +0000 UTC m=+717.724899581" watchObservedRunningTime="2026-05-20 19:11:00.179407457 +0000 UTC m=+717.725366312" May 20 19:11:06.188082 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:06.188026 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-kx5fw" May 20 19:11:12.572353 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.572311 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68fb5d86bc-bsjnv"] May 20 19:11:12.593123 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.593086 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68fb5d86bc-bsjnv"] May 20 19:11:12.593309 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.593228 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" May 20 19:11:12.600676 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.600651 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" May 20 19:11:12.601307 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.601285 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zfwv9\"" May 20 19:11:12.709069 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.709030 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg9m4\" (UniqueName: \"kubernetes.io/projected/bc247ea6-7811-49bd-8b65-04d6211e5464-kube-api-access-tg9m4\") pod \"maas-controller-68fb5d86bc-bsjnv\" (UID: \"bc247ea6-7811-49bd-8b65-04d6211e5464\") " pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" May 20 19:11:12.754133 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.754100 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-854c7f8cf5-87khj"] May 20 19:11:12.756289 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.756273 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854c7f8cf5-87khj" May 20 19:11:12.770877 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.770848 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-854c7f8cf5-87khj"] May 20 19:11:12.809864 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.809827 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tg9m4\" (UniqueName: \"kubernetes.io/projected/bc247ea6-7811-49bd-8b65-04d6211e5464-kube-api-access-tg9m4\") pod \"maas-controller-68fb5d86bc-bsjnv\" (UID: \"bc247ea6-7811-49bd-8b65-04d6211e5464\") " pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" May 20 19:11:12.810019 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.809872 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77vcd\" (UniqueName: \"kubernetes.io/projected/91a4c0b7-565e-4583-82b3-1e837beaf9aa-kube-api-access-77vcd\") pod \"maas-controller-854c7f8cf5-87khj\" (UID: \"91a4c0b7-565e-4583-82b3-1e837beaf9aa\") " pod="opendatahub/maas-controller-854c7f8cf5-87khj" May 20 19:11:12.819324 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.819296 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg9m4\" (UniqueName: \"kubernetes.io/projected/bc247ea6-7811-49bd-8b65-04d6211e5464-kube-api-access-tg9m4\") pod \"maas-controller-68fb5d86bc-bsjnv\" (UID: \"bc247ea6-7811-49bd-8b65-04d6211e5464\") " pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" May 20 19:11:12.884741 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.884658 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68fb5d86bc-bsjnv"] May 20 19:11:12.884916 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.884903 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" May 20 19:11:12.910663 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.910635 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6b7f69f554-8fqhp"] May 20 19:11:12.910969 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.910949 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-77vcd\" (UniqueName: \"kubernetes.io/projected/91a4c0b7-565e-4583-82b3-1e837beaf9aa-kube-api-access-77vcd\") pod \"maas-controller-854c7f8cf5-87khj\" (UID: \"91a4c0b7-565e-4583-82b3-1e837beaf9aa\") " pod="opendatahub/maas-controller-854c7f8cf5-87khj" May 20 19:11:12.913442 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.913425 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b7f69f554-8fqhp" May 20 19:11:12.919372 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.919343 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77vcd\" (UniqueName: \"kubernetes.io/projected/91a4c0b7-565e-4583-82b3-1e837beaf9aa-kube-api-access-77vcd\") pod \"maas-controller-854c7f8cf5-87khj\" (UID: \"91a4c0b7-565e-4583-82b3-1e837beaf9aa\") " pod="opendatahub/maas-controller-854c7f8cf5-87khj" May 20 19:11:12.920934 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:12.920911 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b7f69f554-8fqhp"] May 20 19:11:13.012303 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:13.012270 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4psv\" (UniqueName: \"kubernetes.io/projected/f763e3f1-d852-4e90-ae30-63b7f5f4f81d-kube-api-access-n4psv\") pod \"maas-controller-6b7f69f554-8fqhp\" (UID: \"f763e3f1-d852-4e90-ae30-63b7f5f4f81d\") " pod="opendatahub/maas-controller-6b7f69f554-8fqhp" May 20 19:11:13.018233 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:13.018092 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68fb5d86bc-bsjnv"] May 20 19:11:13.020758 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:11:13.020729 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc247ea6_7811_49bd_8b65_04d6211e5464.slice/crio-1dccb181f9f92ec8f2a4dd130f9bba6248296a461f6ccd4d98d19cf34cde4860 WatchSource:0}: Error finding container 1dccb181f9f92ec8f2a4dd130f9bba6248296a461f6ccd4d98d19cf34cde4860: Status 404 returned error can't find the container with id 1dccb181f9f92ec8f2a4dd130f9bba6248296a461f6ccd4d98d19cf34cde4860 May 20 19:11:13.071005 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:13.070974 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854c7f8cf5-87khj" May 20 19:11:13.112888 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:13.112859 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n4psv\" (UniqueName: \"kubernetes.io/projected/f763e3f1-d852-4e90-ae30-63b7f5f4f81d-kube-api-access-n4psv\") pod \"maas-controller-6b7f69f554-8fqhp\" (UID: \"f763e3f1-d852-4e90-ae30-63b7f5f4f81d\") " pod="opendatahub/maas-controller-6b7f69f554-8fqhp" May 20 19:11:13.121963 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:13.121915 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4psv\" (UniqueName: \"kubernetes.io/projected/f763e3f1-d852-4e90-ae30-63b7f5f4f81d-kube-api-access-n4psv\") pod \"maas-controller-6b7f69f554-8fqhp\" (UID: \"f763e3f1-d852-4e90-ae30-63b7f5f4f81d\") " pod="opendatahub/maas-controller-6b7f69f554-8fqhp" May 20 19:11:13.197248 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:13.197190 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" event={"ID":"bc247ea6-7811-49bd-8b65-04d6211e5464","Type":"ContainerStarted","Data":"1dccb181f9f92ec8f2a4dd130f9bba6248296a461f6ccd4d98d19cf34cde4860"} May 20 19:11:13.203993 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:13.203884 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-854c7f8cf5-87khj"] May 20 19:11:13.206647 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:11:13.206619 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a4c0b7_565e_4583_82b3_1e837beaf9aa.slice/crio-eadd60448f6a5909c321d2b46f9a4db81cd20eefe8057314e6150de843cc50f9 WatchSource:0}: Error finding container eadd60448f6a5909c321d2b46f9a4db81cd20eefe8057314e6150de843cc50f9: Status 404 returned error can't find the container with id eadd60448f6a5909c321d2b46f9a4db81cd20eefe8057314e6150de843cc50f9 May 20 19:11:13.237067 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:13.237034 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b7f69f554-8fqhp" May 20 19:11:13.381781 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:11:13.381751 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf763e3f1_d852_4e90_ae30_63b7f5f4f81d.slice/crio-243bd04d93ae4a525762332e341fd19b448af9f714027e8acbfb961a7cc13904 WatchSource:0}: Error finding container 243bd04d93ae4a525762332e341fd19b448af9f714027e8acbfb961a7cc13904: Status 404 returned error can't find the container with id 243bd04d93ae4a525762332e341fd19b448af9f714027e8acbfb961a7cc13904 May 20 19:11:13.382620 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:13.382586 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6b7f69f554-8fqhp"] May 20 19:11:14.205544 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:14.205480 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b7f69f554-8fqhp" event={"ID":"f763e3f1-d852-4e90-ae30-63b7f5f4f81d","Type":"ContainerStarted","Data":"243bd04d93ae4a525762332e341fd19b448af9f714027e8acbfb961a7cc13904"} May 20 19:11:14.207954 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:14.207892 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854c7f8cf5-87khj" event={"ID":"91a4c0b7-565e-4583-82b3-1e837beaf9aa","Type":"ContainerStarted","Data":"eadd60448f6a5909c321d2b46f9a4db81cd20eefe8057314e6150de843cc50f9"} May 20 19:11:17.218528 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.218491 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b7f69f554-8fqhp" event={"ID":"f763e3f1-d852-4e90-ae30-63b7f5f4f81d","Type":"ContainerStarted","Data":"ab1c53aa54b54188780c3581d7590d0362e6016bca08ad82be51d21be334fc59"} May 20 19:11:17.219018 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.218611 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6b7f69f554-8fqhp" May 20 19:11:17.219841 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.219817 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854c7f8cf5-87khj" event={"ID":"91a4c0b7-565e-4583-82b3-1e837beaf9aa","Type":"ContainerStarted","Data":"f376f4ea70d1688a17b77c3e47c4687ae8c7d6eb35cbc0100ae22ab21c3d8d87"} May 20 19:11:17.219954 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.219877 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-854c7f8cf5-87khj" May 20 19:11:17.220988 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.220968 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" event={"ID":"bc247ea6-7811-49bd-8b65-04d6211e5464","Type":"ContainerStarted","Data":"a7120d5088235f7cea45e86bd43563bdff7b4f6348b5e4b6c69ead543ee3df4b"} May 20 19:11:17.221082 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.221053 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" May 20 19:11:17.221082 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.221049 2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" podUID="bc247ea6-7811-49bd-8b65-04d6211e5464" containerName="manager" containerID="cri-o://a7120d5088235f7cea45e86bd43563bdff7b4f6348b5e4b6c69ead543ee3df4b" gracePeriod=30 May 20 19:11:17.241070 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.241029 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-6b7f69f554-8fqhp" podStartSLOduration=2.161286658 podStartE2EDuration="5.241016929s" podCreationTimestamp="2026-05-20 19:11:12 +0000 UTC" firstStartedPulling="2026-05-20 19:11:13.383132919 +0000 UTC m=+730.929091753" lastFinishedPulling="2026-05-20 19:11:16.46286318 +0000 UTC m=+734.008822024" observedRunningTime="2026-05-20 19:11:17.240147551 +0000 UTC m=+734.786106410" watchObservedRunningTime="2026-05-20 19:11:17.241016929 +0000 UTC m=+734.786975784" May 20 19:11:17.267426 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.267373 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" podStartSLOduration=1.832062263 podStartE2EDuration="5.267360093s" podCreationTimestamp="2026-05-20 19:11:12 +0000 UTC" firstStartedPulling="2026-05-20 19:11:13.022174664 +0000 UTC m=+730.568133498" lastFinishedPulling="2026-05-20 19:11:16.457472495 +0000 UTC m=+734.003431328" observedRunningTime="2026-05-20 19:11:17.263969196 +0000 UTC m=+734.809928051" watchObservedRunningTime="2026-05-20 19:11:17.267360093 +0000 UTC m=+734.813318948" May 20 19:11:17.469635 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.469582 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" May 20 19:11:17.492621 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.492573 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-854c7f8cf5-87khj" podStartSLOduration=2.246798772 podStartE2EDuration="5.492558105s" podCreationTimestamp="2026-05-20 19:11:12 +0000 UTC" firstStartedPulling="2026-05-20 19:11:13.207961265 +0000 UTC m=+730.753920100" lastFinishedPulling="2026-05-20 19:11:16.4537206 +0000 UTC m=+733.999679433" observedRunningTime="2026-05-20 19:11:17.332306107 +0000 UTC m=+734.878264974" watchObservedRunningTime="2026-05-20 19:11:17.492558105 +0000 UTC m=+735.038516997" May 20 19:11:17.557400 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.557355 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tg9m4\" (UniqueName: \"kubernetes.io/projected/bc247ea6-7811-49bd-8b65-04d6211e5464-kube-api-access-tg9m4\") pod \"bc247ea6-7811-49bd-8b65-04d6211e5464\" (UID: \"bc247ea6-7811-49bd-8b65-04d6211e5464\") " May 20 19:11:17.559541 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.559512 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc247ea6-7811-49bd-8b65-04d6211e5464-kube-api-access-tg9m4" (OuterVolumeSpecName: "kube-api-access-tg9m4") pod "bc247ea6-7811-49bd-8b65-04d6211e5464" (UID: "bc247ea6-7811-49bd-8b65-04d6211e5464"). InnerVolumeSpecName "kube-api-access-tg9m4". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 19:11:17.658059 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:17.658027 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg9m4\" (UniqueName: \"kubernetes.io/projected/bc247ea6-7811-49bd-8b65-04d6211e5464-kube-api-access-tg9m4\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:11:18.225361 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:18.225324 2575 generic.go:358] "Generic (PLEG): container finished" podID="bc247ea6-7811-49bd-8b65-04d6211e5464" containerID="a7120d5088235f7cea45e86bd43563bdff7b4f6348b5e4b6c69ead543ee3df4b" exitCode=0 May 20 19:11:18.225777 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:18.225389 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" May 20 19:11:18.225777 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:18.225407 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" event={"ID":"bc247ea6-7811-49bd-8b65-04d6211e5464","Type":"ContainerDied","Data":"a7120d5088235f7cea45e86bd43563bdff7b4f6348b5e4b6c69ead543ee3df4b"} May 20 19:11:18.225777 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:18.225452 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68fb5d86bc-bsjnv" event={"ID":"bc247ea6-7811-49bd-8b65-04d6211e5464","Type":"ContainerDied","Data":"1dccb181f9f92ec8f2a4dd130f9bba6248296a461f6ccd4d98d19cf34cde4860"} May 20 19:11:18.225777 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:18.225475 2575 scope.go:117] "RemoveContainer" containerID="a7120d5088235f7cea45e86bd43563bdff7b4f6348b5e4b6c69ead543ee3df4b" May 20 19:11:18.234060 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:18.234040 2575 scope.go:117] "RemoveContainer" containerID="a7120d5088235f7cea45e86bd43563bdff7b4f6348b5e4b6c69ead543ee3df4b" May 20 19:11:18.234314 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:11:18.234297 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7120d5088235f7cea45e86bd43563bdff7b4f6348b5e4b6c69ead543ee3df4b\": container with ID starting with a7120d5088235f7cea45e86bd43563bdff7b4f6348b5e4b6c69ead543ee3df4b not found: ID does not exist" containerID="a7120d5088235f7cea45e86bd43563bdff7b4f6348b5e4b6c69ead543ee3df4b" May 20 19:11:18.234364 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:18.234322 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7120d5088235f7cea45e86bd43563bdff7b4f6348b5e4b6c69ead543ee3df4b"} err="failed to get container status \"a7120d5088235f7cea45e86bd43563bdff7b4f6348b5e4b6c69ead543ee3df4b\": rpc error: code = NotFound desc = could not find container \"a7120d5088235f7cea45e86bd43563bdff7b4f6348b5e4b6c69ead543ee3df4b\": container with ID starting with a7120d5088235f7cea45e86bd43563bdff7b4f6348b5e4b6c69ead543ee3df4b not found: ID does not exist" May 20 19:11:18.247965 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:18.247940 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68fb5d86bc-bsjnv"] May 20 19:11:18.252991 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:18.252970 2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68fb5d86bc-bsjnv"] May 20 19:11:19.089939 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:19.089908 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc247ea6-7811-49bd-8b65-04d6211e5464" path="/var/lib/kubelet/pods/bc247ea6-7811-49bd-8b65-04d6211e5464/volumes" May 20 19:11:28.230201 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.230168 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-854c7f8cf5-87khj" May 20 19:11:28.230594 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.230434 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6b7f69f554-8fqhp" May 20 19:11:28.296443 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.296412 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-854c7f8cf5-87khj"] May 20 19:11:28.296642 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.296621 2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-854c7f8cf5-87khj" podUID="91a4c0b7-565e-4583-82b3-1e837beaf9aa" containerName="manager" containerID="cri-o://f376f4ea70d1688a17b77c3e47c4687ae8c7d6eb35cbc0100ae22ab21c3d8d87" gracePeriod=30 May 20 19:11:28.488951 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.488861 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-549f87f94d-vbq5k"] May 20 19:11:28.489347 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.489332 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc247ea6-7811-49bd-8b65-04d6211e5464" containerName="manager" May 20 19:11:28.489405 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.489350 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc247ea6-7811-49bd-8b65-04d6211e5464" containerName="manager" May 20 19:11:28.489405 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.489398 2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="bc247ea6-7811-49bd-8b65-04d6211e5464" containerName="manager" May 20 19:11:28.492244 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.492224 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-549f87f94d-vbq5k" May 20 19:11:28.504622 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.504592 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-549f87f94d-vbq5k"] May 20 19:11:28.536812 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.536788 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854c7f8cf5-87khj" May 20 19:11:28.648424 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.648394 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77vcd\" (UniqueName: \"kubernetes.io/projected/91a4c0b7-565e-4583-82b3-1e837beaf9aa-kube-api-access-77vcd\") pod \"91a4c0b7-565e-4583-82b3-1e837beaf9aa\" (UID: \"91a4c0b7-565e-4583-82b3-1e837beaf9aa\") " May 20 19:11:28.648592 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.648540 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5754\" (UniqueName: \"kubernetes.io/projected/36afe091-344b-4e95-b97f-38d87396e820-kube-api-access-n5754\") pod \"maas-controller-549f87f94d-vbq5k\" (UID: \"36afe091-344b-4e95-b97f-38d87396e820\") " pod="opendatahub/maas-controller-549f87f94d-vbq5k" May 20 19:11:28.650497 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.650471 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91a4c0b7-565e-4583-82b3-1e837beaf9aa-kube-api-access-77vcd" (OuterVolumeSpecName: "kube-api-access-77vcd") pod "91a4c0b7-565e-4583-82b3-1e837beaf9aa" (UID: "91a4c0b7-565e-4583-82b3-1e837beaf9aa"). InnerVolumeSpecName "kube-api-access-77vcd". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 19:11:28.749602 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.749513 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n5754\" (UniqueName: \"kubernetes.io/projected/36afe091-344b-4e95-b97f-38d87396e820-kube-api-access-n5754\") pod \"maas-controller-549f87f94d-vbq5k\" (UID: \"36afe091-344b-4e95-b97f-38d87396e820\") " pod="opendatahub/maas-controller-549f87f94d-vbq5k" May 20 19:11:28.749602 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.749577 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77vcd\" (UniqueName: \"kubernetes.io/projected/91a4c0b7-565e-4583-82b3-1e837beaf9aa-kube-api-access-77vcd\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:11:28.758476 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.758453 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5754\" (UniqueName: \"kubernetes.io/projected/36afe091-344b-4e95-b97f-38d87396e820-kube-api-access-n5754\") pod \"maas-controller-549f87f94d-vbq5k\" (UID: \"36afe091-344b-4e95-b97f-38d87396e820\") " pod="opendatahub/maas-controller-549f87f94d-vbq5k" May 20 19:11:28.804438 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.804402 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-549f87f94d-vbq5k" May 20 19:11:28.928185 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:28.928159 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-549f87f94d-vbq5k"] May 20 19:11:28.930704 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:11:28.930678 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36afe091_344b_4e95_b97f_38d87396e820.slice/crio-32a88d9774e27387facaba71320fb1e1a14fad7fce66e8c52a700c2e3b9516f8 WatchSource:0}: Error finding container 32a88d9774e27387facaba71320fb1e1a14fad7fce66e8c52a700c2e3b9516f8: Status 404 returned error can't find the container with id 32a88d9774e27387facaba71320fb1e1a14fad7fce66e8c52a700c2e3b9516f8 May 20 19:11:29.263387 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:29.263351 2575 generic.go:358] "Generic (PLEG): container finished" podID="91a4c0b7-565e-4583-82b3-1e837beaf9aa" containerID="f376f4ea70d1688a17b77c3e47c4687ae8c7d6eb35cbc0100ae22ab21c3d8d87" exitCode=0 May 20 19:11:29.263818 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:29.263421 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-854c7f8cf5-87khj" May 20 19:11:29.263818 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:29.263426 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854c7f8cf5-87khj" event={"ID":"91a4c0b7-565e-4583-82b3-1e837beaf9aa","Type":"ContainerDied","Data":"f376f4ea70d1688a17b77c3e47c4687ae8c7d6eb35cbc0100ae22ab21c3d8d87"} May 20 19:11:29.263818 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:29.263459 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-854c7f8cf5-87khj" event={"ID":"91a4c0b7-565e-4583-82b3-1e837beaf9aa","Type":"ContainerDied","Data":"eadd60448f6a5909c321d2b46f9a4db81cd20eefe8057314e6150de843cc50f9"} May 20 19:11:29.263818 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:29.263476 2575 scope.go:117] "RemoveContainer" containerID="f376f4ea70d1688a17b77c3e47c4687ae8c7d6eb35cbc0100ae22ab21c3d8d87" May 20 19:11:29.264587 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:29.264564 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-549f87f94d-vbq5k" event={"ID":"36afe091-344b-4e95-b97f-38d87396e820","Type":"ContainerStarted","Data":"32a88d9774e27387facaba71320fb1e1a14fad7fce66e8c52a700c2e3b9516f8"} May 20 19:11:29.276088 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:29.276072 2575 scope.go:117] "RemoveContainer" containerID="f376f4ea70d1688a17b77c3e47c4687ae8c7d6eb35cbc0100ae22ab21c3d8d87" May 20 19:11:29.276385 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:11:29.276362 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f376f4ea70d1688a17b77c3e47c4687ae8c7d6eb35cbc0100ae22ab21c3d8d87\": container with ID starting with f376f4ea70d1688a17b77c3e47c4687ae8c7d6eb35cbc0100ae22ab21c3d8d87 not found: ID does not exist" containerID="f376f4ea70d1688a17b77c3e47c4687ae8c7d6eb35cbc0100ae22ab21c3d8d87" May 20 19:11:29.276472 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:29.276391 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f376f4ea70d1688a17b77c3e47c4687ae8c7d6eb35cbc0100ae22ab21c3d8d87"} err="failed to get container status \"f376f4ea70d1688a17b77c3e47c4687ae8c7d6eb35cbc0100ae22ab21c3d8d87\": rpc error: code = NotFound desc = could not find container \"f376f4ea70d1688a17b77c3e47c4687ae8c7d6eb35cbc0100ae22ab21c3d8d87\": container with ID starting with f376f4ea70d1688a17b77c3e47c4687ae8c7d6eb35cbc0100ae22ab21c3d8d87 not found: ID does not exist" May 20 19:11:29.288259 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:29.288234 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-854c7f8cf5-87khj"] May 20 19:11:29.289718 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:29.289697 2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-854c7f8cf5-87khj"] May 20 19:11:30.269882 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:30.269849 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-549f87f94d-vbq5k" event={"ID":"36afe091-344b-4e95-b97f-38d87396e820","Type":"ContainerStarted","Data":"9c0f593d10ecc9d46bb6e21fb8f5708cc8c00b3c0f4865503df596a2f3e163e0"} May 20 19:11:30.270380 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:30.269986 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-549f87f94d-vbq5k" May 20 19:11:30.288528 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:30.288476 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-549f87f94d-vbq5k" podStartSLOduration=1.9432549749999999 podStartE2EDuration="2.288461019s" podCreationTimestamp="2026-05-20 19:11:28 +0000 UTC" firstStartedPulling="2026-05-20 19:11:28.93185793 +0000 UTC m=+746.477816763" lastFinishedPulling="2026-05-20 19:11:29.277063972 +0000 UTC m=+746.823022807" observedRunningTime="2026-05-20 19:11:30.287048246 +0000 UTC m=+747.833007106" watchObservedRunningTime="2026-05-20 19:11:30.288461019 +0000 UTC m=+747.834419873" May 20 19:11:31.089407 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:31.089377 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91a4c0b7-565e-4583-82b3-1e837beaf9aa" path="/var/lib/kubelet/pods/91a4c0b7-565e-4583-82b3-1e837beaf9aa/volumes" May 20 19:11:41.278765 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:41.278736 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-549f87f94d-vbq5k" May 20 19:11:41.338358 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:41.338325 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b7f69f554-8fqhp"] May 20 19:11:41.338577 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:41.338554 2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-6b7f69f554-8fqhp" podUID="f763e3f1-d852-4e90-ae30-63b7f5f4f81d" containerName="manager" containerID="cri-o://ab1c53aa54b54188780c3581d7590d0362e6016bca08ad82be51d21be334fc59" gracePeriod=30 May 20 19:11:41.584530 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:41.584505 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b7f69f554-8fqhp" May 20 19:11:41.761144 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:41.761106 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4psv\" (UniqueName: \"kubernetes.io/projected/f763e3f1-d852-4e90-ae30-63b7f5f4f81d-kube-api-access-n4psv\") pod \"f763e3f1-d852-4e90-ae30-63b7f5f4f81d\" (UID: \"f763e3f1-d852-4e90-ae30-63b7f5f4f81d\") " May 20 19:11:41.763218 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:41.763175 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f763e3f1-d852-4e90-ae30-63b7f5f4f81d-kube-api-access-n4psv" (OuterVolumeSpecName: "kube-api-access-n4psv") pod "f763e3f1-d852-4e90-ae30-63b7f5f4f81d" (UID: "f763e3f1-d852-4e90-ae30-63b7f5f4f81d"). InnerVolumeSpecName "kube-api-access-n4psv". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 19:11:41.861758 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:41.861671 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4psv\" (UniqueName: \"kubernetes.io/projected/f763e3f1-d852-4e90-ae30-63b7f5f4f81d-kube-api-access-n4psv\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:11:42.309260 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:42.309216 2575 generic.go:358] "Generic (PLEG): container finished" podID="f763e3f1-d852-4e90-ae30-63b7f5f4f81d" containerID="ab1c53aa54b54188780c3581d7590d0362e6016bca08ad82be51d21be334fc59" exitCode=0 May 20 19:11:42.309680 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:42.309285 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b7f69f554-8fqhp" event={"ID":"f763e3f1-d852-4e90-ae30-63b7f5f4f81d","Type":"ContainerDied","Data":"ab1c53aa54b54188780c3581d7590d0362e6016bca08ad82be51d21be334fc59"} May 20 19:11:42.309680 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:42.309285 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6b7f69f554-8fqhp" May 20 19:11:42.309680 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:42.309320 2575 scope.go:117] "RemoveContainer" containerID="ab1c53aa54b54188780c3581d7590d0362e6016bca08ad82be51d21be334fc59" May 20 19:11:42.309680 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:42.309310 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6b7f69f554-8fqhp" event={"ID":"f763e3f1-d852-4e90-ae30-63b7f5f4f81d","Type":"ContainerDied","Data":"243bd04d93ae4a525762332e341fd19b448af9f714027e8acbfb961a7cc13904"} May 20 19:11:42.317752 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:42.317725 2575 scope.go:117] "RemoveContainer" containerID="ab1c53aa54b54188780c3581d7590d0362e6016bca08ad82be51d21be334fc59" May 20 19:11:42.318014 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:11:42.317993 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab1c53aa54b54188780c3581d7590d0362e6016bca08ad82be51d21be334fc59\": container with ID starting with ab1c53aa54b54188780c3581d7590d0362e6016bca08ad82be51d21be334fc59 not found: ID does not exist" containerID="ab1c53aa54b54188780c3581d7590d0362e6016bca08ad82be51d21be334fc59" May 20 19:11:42.318057 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:42.318023 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab1c53aa54b54188780c3581d7590d0362e6016bca08ad82be51d21be334fc59"} err="failed to get container status \"ab1c53aa54b54188780c3581d7590d0362e6016bca08ad82be51d21be334fc59\": rpc error: code = NotFound desc = could not find container \"ab1c53aa54b54188780c3581d7590d0362e6016bca08ad82be51d21be334fc59\": container with ID starting with ab1c53aa54b54188780c3581d7590d0362e6016bca08ad82be51d21be334fc59 not found: ID does not exist" May 20 19:11:42.332164 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:42.332135 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6b7f69f554-8fqhp"] May 20 19:11:42.340572 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:42.340547 2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6b7f69f554-8fqhp"] May 20 19:11:43.088926 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:11:43.088897 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f763e3f1-d852-4e90-ae30-63b7f5f4f81d" path="/var/lib/kubelet/pods/f763e3f1-d852-4e90-ae30-63b7f5f4f81d/volumes" May 20 19:12:01.343249 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.343199 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5d4c4dbf77-prb6g"] May 20 19:12:01.343717 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.343509 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f763e3f1-d852-4e90-ae30-63b7f5f4f81d" containerName="manager" May 20 19:12:01.343717 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.343521 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f763e3f1-d852-4e90-ae30-63b7f5f4f81d" containerName="manager" May 20 19:12:01.343717 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.343542 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91a4c0b7-565e-4583-82b3-1e837beaf9aa" containerName="manager" May 20 19:12:01.343717 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.343547 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="91a4c0b7-565e-4583-82b3-1e837beaf9aa" containerName="manager" May 20 19:12:01.343717 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.343596 2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="91a4c0b7-565e-4583-82b3-1e837beaf9aa" containerName="manager" May 20 19:12:01.343717 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.343606 2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="f763e3f1-d852-4e90-ae30-63b7f5f4f81d" containerName="manager" May 20 19:12:01.345876 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.345859 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5d4c4dbf77-prb6g" May 20 19:12:01.348588 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.348569 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8slc6\"" May 20 19:12:01.348682 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.348592 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" May 20 19:12:01.373028 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.372982 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5d4c4dbf77-prb6g"] May 20 19:12:01.417861 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.417826 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/537eb9cc-53ef-4363-9fb7-20e0cfc7edbc-maas-api-tls\") pod \"maas-api-5d4c4dbf77-prb6g\" (UID: \"537eb9cc-53ef-4363-9fb7-20e0cfc7edbc\") " pod="opendatahub/maas-api-5d4c4dbf77-prb6g" May 20 19:12:01.417861 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.417866 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2l9q\" (UniqueName: \"kubernetes.io/projected/537eb9cc-53ef-4363-9fb7-20e0cfc7edbc-kube-api-access-v2l9q\") pod \"maas-api-5d4c4dbf77-prb6g\" (UID: \"537eb9cc-53ef-4363-9fb7-20e0cfc7edbc\") " pod="opendatahub/maas-api-5d4c4dbf77-prb6g" May 20 19:12:01.519119 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.519086 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/537eb9cc-53ef-4363-9fb7-20e0cfc7edbc-maas-api-tls\") pod \"maas-api-5d4c4dbf77-prb6g\" (UID: \"537eb9cc-53ef-4363-9fb7-20e0cfc7edbc\") " pod="opendatahub/maas-api-5d4c4dbf77-prb6g" May 20 19:12:01.519297 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.519129 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v2l9q\" (UniqueName: \"kubernetes.io/projected/537eb9cc-53ef-4363-9fb7-20e0cfc7edbc-kube-api-access-v2l9q\") pod \"maas-api-5d4c4dbf77-prb6g\" (UID: \"537eb9cc-53ef-4363-9fb7-20e0cfc7edbc\") " pod="opendatahub/maas-api-5d4c4dbf77-prb6g" May 20 19:12:01.521366 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.521347 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/537eb9cc-53ef-4363-9fb7-20e0cfc7edbc-maas-api-tls\") pod \"maas-api-5d4c4dbf77-prb6g\" (UID: \"537eb9cc-53ef-4363-9fb7-20e0cfc7edbc\") " pod="opendatahub/maas-api-5d4c4dbf77-prb6g" May 20 19:12:01.529546 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.529524 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2l9q\" (UniqueName: \"kubernetes.io/projected/537eb9cc-53ef-4363-9fb7-20e0cfc7edbc-kube-api-access-v2l9q\") pod \"maas-api-5d4c4dbf77-prb6g\" (UID: \"537eb9cc-53ef-4363-9fb7-20e0cfc7edbc\") " pod="opendatahub/maas-api-5d4c4dbf77-prb6g" May 20 19:12:01.655833 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.655754 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5d4c4dbf77-prb6g" May 20 19:12:01.702892 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.702700 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-jggd8"] May 20 19:12:01.706788 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.706763 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-jggd8" May 20 19:12:01.710217 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.710185 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" May 20 19:12:01.727345 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.727310 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-jggd8"] May 20 19:12:01.803130 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.803100 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5d4c4dbf77-prb6g"] May 20 19:12:01.804927 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:12:01.804900 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod537eb9cc_53ef_4363_9fb7_20e0cfc7edbc.slice/crio-46fe470fd76f9e8d746fe1e55f46656238ee06a1f9468f8328ec51ff6dd6166a WatchSource:0}: Error finding container 46fe470fd76f9e8d746fe1e55f46656238ee06a1f9468f8328ec51ff6dd6166a: Status 404 returned error can't find the container with id 46fe470fd76f9e8d746fe1e55f46656238ee06a1f9468f8328ec51ff6dd6166a May 20 19:12:01.835955 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.835920 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qbwx\" (UniqueName: \"kubernetes.io/projected/40dfee4e-ef27-4500-b206-984941dcb97f-kube-api-access-7qbwx\") pod \"payload-processing-58b6c8fdc7-jggd8\" (UID: \"40dfee4e-ef27-4500-b206-984941dcb97f\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-jggd8" May 20 19:12:01.936863 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.936769 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7qbwx\" (UniqueName: \"kubernetes.io/projected/40dfee4e-ef27-4500-b206-984941dcb97f-kube-api-access-7qbwx\") pod \"payload-processing-58b6c8fdc7-jggd8\" (UID: \"40dfee4e-ef27-4500-b206-984941dcb97f\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-jggd8" May 20 19:12:01.949526 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:01.949492 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qbwx\" (UniqueName: \"kubernetes.io/projected/40dfee4e-ef27-4500-b206-984941dcb97f-kube-api-access-7qbwx\") pod \"payload-processing-58b6c8fdc7-jggd8\" (UID: \"40dfee4e-ef27-4500-b206-984941dcb97f\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-jggd8" May 20 19:12:02.026903 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:02.026871 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-jggd8" May 20 19:12:02.162297 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:02.162269 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-jggd8"] May 20 19:12:02.164517 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:12:02.164487 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40dfee4e_ef27_4500_b206_984941dcb97f.slice/crio-6c7cb9455abdfedcaeaf1b9037c7b111c01e6d48344bd5669fa84e4aff50615c WatchSource:0}: Error finding container 6c7cb9455abdfedcaeaf1b9037c7b111c01e6d48344bd5669fa84e4aff50615c: Status 404 returned error can't find the container with id 6c7cb9455abdfedcaeaf1b9037c7b111c01e6d48344bd5669fa84e4aff50615c May 20 19:12:02.378664 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:02.378610 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-jggd8" event={"ID":"40dfee4e-ef27-4500-b206-984941dcb97f","Type":"ContainerStarted","Data":"6c7cb9455abdfedcaeaf1b9037c7b111c01e6d48344bd5669fa84e4aff50615c"} May 20 19:12:02.379863 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:02.379838 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5d4c4dbf77-prb6g" event={"ID":"537eb9cc-53ef-4363-9fb7-20e0cfc7edbc","Type":"ContainerStarted","Data":"46fe470fd76f9e8d746fe1e55f46656238ee06a1f9468f8328ec51ff6dd6166a"} May 20 19:12:03.697631 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:03.697613 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" May 20 19:12:04.390452 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:04.390407 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-jggd8" event={"ID":"40dfee4e-ef27-4500-b206-984941dcb97f","Type":"ContainerStarted","Data":"a627d24cea01bdedecf42a178116b873330797998fff84dfbb7dc5b359e78d53"} May 20 19:12:04.390656 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:04.390535 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-jggd8" May 20 19:12:04.391715 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:04.391689 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5d4c4dbf77-prb6g" event={"ID":"537eb9cc-53ef-4363-9fb7-20e0cfc7edbc","Type":"ContainerStarted","Data":"e49222b00386edc16a23a052e40141a8e362b4b55653b4ace4f9b88705667f70"} May 20 19:12:04.391873 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:04.391857 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5d4c4dbf77-prb6g" May 20 19:12:04.412450 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:04.412400 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-jggd8" podStartSLOduration=1.884205323 podStartE2EDuration="3.412387014s" podCreationTimestamp="2026-05-20 19:12:01 +0000 UTC" firstStartedPulling="2026-05-20 19:12:02.166656481 +0000 UTC m=+779.712615327" lastFinishedPulling="2026-05-20 19:12:03.694838185 +0000 UTC m=+781.240797018" observedRunningTime="2026-05-20 19:12:04.409814758 +0000 UTC m=+781.955773624" watchObservedRunningTime="2026-05-20 19:12:04.412387014 +0000 UTC m=+781.958345907" May 20 19:12:04.444837 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:04.441380 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-5d4c4dbf77-prb6g" podStartSLOduration=1.5546404790000001 podStartE2EDuration="3.441358834s" podCreationTimestamp="2026-05-20 19:12:01 +0000 UTC" firstStartedPulling="2026-05-20 19:12:01.806100304 +0000 UTC m=+779.352059138" lastFinishedPulling="2026-05-20 19:12:03.69281866 +0000 UTC m=+781.238777493" observedRunningTime="2026-05-20 19:12:04.439962927 +0000 UTC m=+781.985921783" watchObservedRunningTime="2026-05-20 19:12:04.441358834 +0000 UTC m=+781.987317691" May 20 19:12:10.400496 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:10.400467 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5d4c4dbf77-prb6g" May 20 19:12:15.397386 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:15.397352 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-jggd8" May 20 19:12:26.804986 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:26.804903 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4"] May 20 19:12:26.808321 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:26.808304 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:26.811293 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:26.811273 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" May 20 19:12:26.812158 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:26.812138 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" May 20 19:12:26.812804 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:26.812786 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" May 20 19:12:26.813173 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:26.813157 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kwgsp\"" May 20 19:12:26.820385 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:26.820352 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4"] May 20 19:12:26.953037 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:26.952998 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9ls4\" (UniqueName: \"kubernetes.io/projected/bd786bb3-fcbc-41da-a19d-b21a0836c109-kube-api-access-r9ls4\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:26.953252 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:26.953054 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bd786bb3-fcbc-41da-a19d-b21a0836c109-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:26.953252 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:26.953152 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd786bb3-fcbc-41da-a19d-b21a0836c109-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:26.953252 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:26.953191 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd786bb3-fcbc-41da-a19d-b21a0836c109-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:26.953252 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:26.953248 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd786bb3-fcbc-41da-a19d-b21a0836c109-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:26.953474 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:26.953302 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd786bb3-fcbc-41da-a19d-b21a0836c109-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.053960 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.053918 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bd786bb3-fcbc-41da-a19d-b21a0836c109-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.054147 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.053979 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd786bb3-fcbc-41da-a19d-b21a0836c109-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.054147 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.054006 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd786bb3-fcbc-41da-a19d-b21a0836c109-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.054147 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.054036 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd786bb3-fcbc-41da-a19d-b21a0836c109-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.054427 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.054399 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bd786bb3-fcbc-41da-a19d-b21a0836c109-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.054597 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.054410 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd786bb3-fcbc-41da-a19d-b21a0836c109-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.054687 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.054500 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd786bb3-fcbc-41da-a19d-b21a0836c109-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.054764 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.054521 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd786bb3-fcbc-41da-a19d-b21a0836c109-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.054928 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.054914 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r9ls4\" (UniqueName: \"kubernetes.io/projected/bd786bb3-fcbc-41da-a19d-b21a0836c109-kube-api-access-r9ls4\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.056819 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.056761 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd786bb3-fcbc-41da-a19d-b21a0836c109-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.057549 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.057529 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd786bb3-fcbc-41da-a19d-b21a0836c109-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.062956 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.062938 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9ls4\" (UniqueName: \"kubernetes.io/projected/bd786bb3-fcbc-41da-a19d-b21a0836c109-kube-api-access-r9ls4\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4\" (UID: \"bd786bb3-fcbc-41da-a19d-b21a0836c109\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.119541 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.119514 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:27.253928 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.253895 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4"] May 20 19:12:27.259038 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:12:27.258998 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd786bb3_fcbc_41da_a19d_b21a0836c109.slice/crio-9852161808c39c795395200d57a3ed4c5460a9e0cc8ee1559170b3ce1dd6b4a3 WatchSource:0}: Error finding container 9852161808c39c795395200d57a3ed4c5460a9e0cc8ee1559170b3ce1dd6b4a3: Status 404 returned error can't find the container with id 9852161808c39c795395200d57a3ed4c5460a9e0cc8ee1559170b3ce1dd6b4a3 May 20 19:12:27.260931 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.260915 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 20 19:12:27.467149 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:27.467065 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" event={"ID":"bd786bb3-fcbc-41da-a19d-b21a0836c109","Type":"ContainerStarted","Data":"9852161808c39c795395200d57a3ed4c5460a9e0cc8ee1559170b3ce1dd6b4a3"} May 20 19:12:33.490313 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:33.490280 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" event={"ID":"bd786bb3-fcbc-41da-a19d-b21a0836c109","Type":"ContainerStarted","Data":"a0712f616c3f959916d33f7b6a19d92a49e79dcdd403dd96a5b1e8b4175afe39"} May 20 19:12:38.507938 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:38.507901 2575 generic.go:358] "Generic (PLEG): container finished" podID="bd786bb3-fcbc-41da-a19d-b21a0836c109" containerID="a0712f616c3f959916d33f7b6a19d92a49e79dcdd403dd96a5b1e8b4175afe39" exitCode=0 May 20 19:12:38.508300 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:38.507974 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" event={"ID":"bd786bb3-fcbc-41da-a19d-b21a0836c109","Type":"ContainerDied","Data":"a0712f616c3f959916d33f7b6a19d92a49e79dcdd403dd96a5b1e8b4175afe39"} May 20 19:12:40.517395 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:40.517354 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" event={"ID":"bd786bb3-fcbc-41da-a19d-b21a0836c109","Type":"ContainerStarted","Data":"175f02d2040409f23ad31b4f1f49599cff53001cacebc630b0c14b1f98fdff59"} May 20 19:12:40.517758 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:40.517577 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:40.542304 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:40.542256 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" podStartSLOduration=2.068725005 podStartE2EDuration="14.542243115s" podCreationTimestamp="2026-05-20 19:12:26 +0000 UTC" firstStartedPulling="2026-05-20 19:12:27.2610491 +0000 UTC m=+804.807007934" lastFinishedPulling="2026-05-20 19:12:39.734567197 +0000 UTC m=+817.280526044" observedRunningTime="2026-05-20 19:12:40.540977241 +0000 UTC m=+818.086936097" watchObservedRunningTime="2026-05-20 19:12:40.542243115 +0000 UTC m=+818.088201969" May 20 19:12:42.525043 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.525013 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g"] May 20 19:12:42.528351 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.528334 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.531490 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.531469 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" May 20 19:12:42.541942 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.541918 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g"] May 20 19:12:42.592461 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.592423 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.592618 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.592474 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.592618 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.592504 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.592618 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.592535 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.592618 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.592598 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.592783 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.592659 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmz5n\" (UniqueName: \"kubernetes.io/projected/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-kube-api-access-lmz5n\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.607076 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.607050 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp"] May 20 19:12:42.610447 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.610431 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.613708 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.613681 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" May 20 19:12:42.622592 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.622569 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp"] May 20 19:12:42.693620 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.693590 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a48aab9b-0846-41f5-b6ec-4664d7375ec8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.693786 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.693654 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.693786 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.693682 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.693786 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.693701 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.693942 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.693841 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.693942 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.693892 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a48aab9b-0846-41f5-b6ec-4664d7375ec8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.694058 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.693940 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a48aab9b-0846-41f5-b6ec-4664d7375ec8-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.694058 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.693970 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.694160 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.694064 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a48aab9b-0846-41f5-b6ec-4664d7375ec8-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.694160 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.694110 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lmz5n\" (UniqueName: \"kubernetes.io/projected/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-kube-api-access-lmz5n\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.694160 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.694135 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a48aab9b-0846-41f5-b6ec-4664d7375ec8-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.694160 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.694135 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.694347 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.694172 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xrhp\" (UniqueName: \"kubernetes.io/projected/a48aab9b-0846-41f5-b6ec-4664d7375ec8-kube-api-access-5xrhp\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.694347 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.694193 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.694347 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.694251 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.696188 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.696167 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.696487 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.696468 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.703133 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.703108 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmz5n\" (UniqueName: \"kubernetes.io/projected/9b041ef4-8ccb-488a-9f16-2a7bcf072cc2-kube-api-access-lmz5n\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-p579g\" (UID: \"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.795115 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.795086 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5xrhp\" (UniqueName: \"kubernetes.io/projected/a48aab9b-0846-41f5-b6ec-4664d7375ec8-kube-api-access-5xrhp\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.795327 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.795130 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a48aab9b-0846-41f5-b6ec-4664d7375ec8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.795327 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.795178 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a48aab9b-0846-41f5-b6ec-4664d7375ec8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.795327 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.795224 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a48aab9b-0846-41f5-b6ec-4664d7375ec8-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.795492 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.795375 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a48aab9b-0846-41f5-b6ec-4664d7375ec8-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.795492 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.795416 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a48aab9b-0846-41f5-b6ec-4664d7375ec8-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.795669 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.795638 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a48aab9b-0846-41f5-b6ec-4664d7375ec8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.795769 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.795737 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a48aab9b-0846-41f5-b6ec-4664d7375ec8-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.795809 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.795788 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a48aab9b-0846-41f5-b6ec-4664d7375ec8-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.797381 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.797355 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a48aab9b-0846-41f5-b6ec-4664d7375ec8-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.797667 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.797645 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a48aab9b-0846-41f5-b6ec-4664d7375ec8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.803652 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.803629 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xrhp\" (UniqueName: \"kubernetes.io/projected/a48aab9b-0846-41f5-b6ec-4664d7375ec8-kube-api-access-5xrhp\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp\" (UID: \"a48aab9b-0846-41f5-b6ec-4664d7375ec8\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.838694 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.838669 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:42.920276 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.920245 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:42.979718 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:42.979648 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g"] May 20 19:12:42.983233 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:12:42.983182 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b041ef4_8ccb_488a_9f16_2a7bcf072cc2.slice/crio-f208e74dcc18379037a2c1a5c92c5c8da0f9fe246e6263e7496c2598feb2f863 WatchSource:0}: Error finding container f208e74dcc18379037a2c1a5c92c5c8da0f9fe246e6263e7496c2598feb2f863: Status 404 returned error can't find the container with id f208e74dcc18379037a2c1a5c92c5c8da0f9fe246e6263e7496c2598feb2f863 May 20 19:12:43.066291 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:43.066264 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp"] May 20 19:12:43.068087 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:12:43.068063 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda48aab9b_0846_41f5_b6ec_4664d7375ec8.slice/crio-8bea4f6db4473727959585c7474b674951a3862b4fb5753e759f75edc5ceb099 WatchSource:0}: Error finding container 8bea4f6db4473727959585c7474b674951a3862b4fb5753e759f75edc5ceb099: Status 404 returned error can't find the container with id 8bea4f6db4473727959585c7474b674951a3862b4fb5753e759f75edc5ceb099 May 20 19:12:43.530104 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:43.530011 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" event={"ID":"a48aab9b-0846-41f5-b6ec-4664d7375ec8","Type":"ContainerStarted","Data":"354b8f562db28ce2f5bea23d0814a22c283b0c6d64bb94736a43db92c472c564"} May 20 19:12:43.530104 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:43.530055 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" event={"ID":"a48aab9b-0846-41f5-b6ec-4664d7375ec8","Type":"ContainerStarted","Data":"8bea4f6db4473727959585c7474b674951a3862b4fb5753e759f75edc5ceb099"} May 20 19:12:43.531614 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:43.531579 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" event={"ID":"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2","Type":"ContainerStarted","Data":"f04598be70f04d7f90ea85b8875c6824fcd99fb193653044ec6e61f702ca7c4c"} May 20 19:12:43.531614 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:43.531608 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" event={"ID":"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2","Type":"ContainerStarted","Data":"f208e74dcc18379037a2c1a5c92c5c8da0f9fe246e6263e7496c2598feb2f863"} May 20 19:12:49.557471 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:49.557433 2575 generic.go:358] "Generic (PLEG): container finished" podID="a48aab9b-0846-41f5-b6ec-4664d7375ec8" containerID="354b8f562db28ce2f5bea23d0814a22c283b0c6d64bb94736a43db92c472c564" exitCode=0 May 20 19:12:49.557904 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:49.557523 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" event={"ID":"a48aab9b-0846-41f5-b6ec-4664d7375ec8","Type":"ContainerDied","Data":"354b8f562db28ce2f5bea23d0814a22c283b0c6d64bb94736a43db92c472c564"} May 20 19:12:49.559622 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:49.559595 2575 generic.go:358] "Generic (PLEG): container finished" podID="9b041ef4-8ccb-488a-9f16-2a7bcf072cc2" containerID="f04598be70f04d7f90ea85b8875c6824fcd99fb193653044ec6e61f702ca7c4c" exitCode=0 May 20 19:12:49.559738 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:49.559647 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" event={"ID":"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2","Type":"ContainerDied","Data":"f04598be70f04d7f90ea85b8875c6824fcd99fb193653044ec6e61f702ca7c4c"} May 20 19:12:50.565762 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:50.565730 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" event={"ID":"a48aab9b-0846-41f5-b6ec-4664d7375ec8","Type":"ContainerStarted","Data":"f5d0de458b7e412822f6d37189ee1e01fa9ff40156aabcc4d78633d2f851fe53"} May 20 19:12:50.566199 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:50.565968 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:12:50.567394 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:50.567371 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" event={"ID":"9b041ef4-8ccb-488a-9f16-2a7bcf072cc2","Type":"ContainerStarted","Data":"2096964a48829f879adcffab334c7afc01d77f18b40c546e4fa867b701aac18f"} May 20 19:12:50.567555 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:50.567539 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:12:50.586828 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:50.586783 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" podStartSLOduration=8.315042212 podStartE2EDuration="8.586766055s" podCreationTimestamp="2026-05-20 19:12:42 +0000 UTC" firstStartedPulling="2026-05-20 19:12:49.558248793 +0000 UTC m=+827.104207635" lastFinishedPulling="2026-05-20 19:12:49.829972638 +0000 UTC m=+827.375931478" observedRunningTime="2026-05-20 19:12:50.585500693 +0000 UTC m=+828.131459550" watchObservedRunningTime="2026-05-20 19:12:50.586766055 +0000 UTC m=+828.132724911" May 20 19:12:50.606576 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:50.606524 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" podStartSLOduration=8.416166344 podStartE2EDuration="8.606512143s" podCreationTimestamp="2026-05-20 19:12:42 +0000 UTC" firstStartedPulling="2026-05-20 19:12:49.560298713 +0000 UTC m=+827.106257546" lastFinishedPulling="2026-05-20 19:12:49.750644499 +0000 UTC m=+827.296603345" observedRunningTime="2026-05-20 19:12:50.605050571 +0000 UTC m=+828.151009427" watchObservedRunningTime="2026-05-20 19:12:50.606512143 +0000 UTC m=+828.152470998" May 20 19:12:51.533771 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:51.533742 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4" May 20 19:12:53.196751 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.196720 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7"] May 20 19:12:53.201001 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.200956 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.204140 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.204119 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" May 20 19:12:53.210403 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.210380 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7"] May 20 19:12:53.289789 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.289759 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/300abe09-50b4-40c9-9764-8883f3fb703a-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.289939 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.289795 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/300abe09-50b4-40c9-9764-8883f3fb703a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.289939 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.289816 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/300abe09-50b4-40c9-9764-8883f3fb703a-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.289939 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.289886 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/300abe09-50b4-40c9-9764-8883f3fb703a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.290066 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.289950 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/300abe09-50b4-40c9-9764-8883f3fb703a-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.290066 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.289975 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn7km\" (UniqueName: \"kubernetes.io/projected/300abe09-50b4-40c9-9764-8883f3fb703a-kube-api-access-pn7km\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.390649 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.390591 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/300abe09-50b4-40c9-9764-8883f3fb703a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.390649 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.390648 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/300abe09-50b4-40c9-9764-8883f3fb703a-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.390954 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.390677 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pn7km\" (UniqueName: \"kubernetes.io/projected/300abe09-50b4-40c9-9764-8883f3fb703a-kube-api-access-pn7km\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.390954 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.390710 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/300abe09-50b4-40c9-9764-8883f3fb703a-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.390954 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.390737 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/300abe09-50b4-40c9-9764-8883f3fb703a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.390954 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.390769 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/300abe09-50b4-40c9-9764-8883f3fb703a-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.391257 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.391229 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/300abe09-50b4-40c9-9764-8883f3fb703a-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.391462 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.391296 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/300abe09-50b4-40c9-9764-8883f3fb703a-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.391462 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.391349 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/300abe09-50b4-40c9-9764-8883f3fb703a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.393507 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.393488 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/300abe09-50b4-40c9-9764-8883f3fb703a-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.393594 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.393536 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/300abe09-50b4-40c9-9764-8883f3fb703a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.399185 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.399160 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn7km\" (UniqueName: \"kubernetes.io/projected/300abe09-50b4-40c9-9764-8883f3fb703a-kube-api-access-pn7km\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7\" (UID: \"300abe09-50b4-40c9-9764-8883f3fb703a\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.512917 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.512844 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:12:53.658744 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:53.658710 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7"] May 20 19:12:53.661500 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:12:53.661468 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod300abe09_50b4_40c9_9764_8883f3fb703a.slice/crio-d345743242877f5329a00b8f7e1c173cc5eee99ccdbc93128c9372cbfe2aa5d9 WatchSource:0}: Error finding container d345743242877f5329a00b8f7e1c173cc5eee99ccdbc93128c9372cbfe2aa5d9: Status 404 returned error can't find the container with id d345743242877f5329a00b8f7e1c173cc5eee99ccdbc93128c9372cbfe2aa5d9 May 20 19:12:54.584910 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:54.584875 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" event={"ID":"300abe09-50b4-40c9-9764-8883f3fb703a","Type":"ContainerStarted","Data":"0f633d9ca950cb8993ba8a0722a0c91195be0f93bbe267fe75a8748dde6638a0"} May 20 19:12:54.584910 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:54.584911 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" event={"ID":"300abe09-50b4-40c9-9764-8883f3fb703a","Type":"ContainerStarted","Data":"d345743242877f5329a00b8f7e1c173cc5eee99ccdbc93128c9372cbfe2aa5d9"} May 20 19:12:59.605727 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:59.605689 2575 generic.go:358] "Generic (PLEG): container finished" podID="300abe09-50b4-40c9-9764-8883f3fb703a" containerID="0f633d9ca950cb8993ba8a0722a0c91195be0f93bbe267fe75a8748dde6638a0" exitCode=0 May 20 19:12:59.606190 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:12:59.605754 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" event={"ID":"300abe09-50b4-40c9-9764-8883f3fb703a","Type":"ContainerDied","Data":"0f633d9ca950cb8993ba8a0722a0c91195be0f93bbe267fe75a8748dde6638a0"} May 20 19:13:00.613198 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:13:00.613161 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" event={"ID":"300abe09-50b4-40c9-9764-8883f3fb703a","Type":"ContainerStarted","Data":"00d1488da859d111ca2df84bbbc45b4088e3cdccfdc8dcaae9e9ca488cfb6baa"} May 20 19:13:00.613590 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:13:00.613399 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:13:00.634020 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:13:00.633970 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" podStartSLOduration=7.36174085 podStartE2EDuration="7.63395676s" podCreationTimestamp="2026-05-20 19:12:53 +0000 UTC" firstStartedPulling="2026-05-20 19:12:59.606539949 +0000 UTC m=+837.152498782" lastFinishedPulling="2026-05-20 19:12:59.878755855 +0000 UTC m=+837.424714692" observedRunningTime="2026-05-20 19:13:00.632344736 +0000 UTC m=+838.178303592" watchObservedRunningTime="2026-05-20 19:13:00.63395676 +0000 UTC m=+838.179915611" May 20 19:13:01.583908 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:13:01.583883 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-p579g" May 20 19:13:01.584740 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:13:01.584712 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp" May 20 19:13:11.628863 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:13:11.628836 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7" May 20 19:14:03.027218 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:03.027125 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:14:03.027218 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:03.027138 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:14:38.556007 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.555973 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-549f87f94d-vbq5k"] May 20 19:14:38.556521 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.556221 2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-549f87f94d-vbq5k" podUID="36afe091-344b-4e95-b97f-38d87396e820" containerName="manager" containerID="cri-o://9c0f593d10ecc9d46bb6e21fb8f5708cc8c00b3c0f4865503df596a2f3e163e0" gracePeriod=30 May 20 19:14:38.796000 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.795974 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-549f87f94d-vbq5k" May 20 19:14:38.899284 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.899201 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5754\" (UniqueName: \"kubernetes.io/projected/36afe091-344b-4e95-b97f-38d87396e820-kube-api-access-n5754\") pod \"36afe091-344b-4e95-b97f-38d87396e820\" (UID: \"36afe091-344b-4e95-b97f-38d87396e820\") " May 20 19:14:38.901103 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.901081 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36afe091-344b-4e95-b97f-38d87396e820-kube-api-access-n5754" (OuterVolumeSpecName: "kube-api-access-n5754") pod "36afe091-344b-4e95-b97f-38d87396e820" (UID: "36afe091-344b-4e95-b97f-38d87396e820"). InnerVolumeSpecName "kube-api-access-n5754". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 19:14:38.947060 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.947032 2575 generic.go:358] "Generic (PLEG): container finished" podID="36afe091-344b-4e95-b97f-38d87396e820" containerID="9c0f593d10ecc9d46bb6e21fb8f5708cc8c00b3c0f4865503df596a2f3e163e0" exitCode=0 May 20 19:14:38.947159 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.947090 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-549f87f94d-vbq5k" May 20 19:14:38.947159 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.947115 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-549f87f94d-vbq5k" event={"ID":"36afe091-344b-4e95-b97f-38d87396e820","Type":"ContainerDied","Data":"9c0f593d10ecc9d46bb6e21fb8f5708cc8c00b3c0f4865503df596a2f3e163e0"} May 20 19:14:38.947159 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.947150 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-549f87f94d-vbq5k" event={"ID":"36afe091-344b-4e95-b97f-38d87396e820","Type":"ContainerDied","Data":"32a88d9774e27387facaba71320fb1e1a14fad7fce66e8c52a700c2e3b9516f8"} May 20 19:14:38.947301 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.947168 2575 scope.go:117] "RemoveContainer" containerID="9c0f593d10ecc9d46bb6e21fb8f5708cc8c00b3c0f4865503df596a2f3e163e0" May 20 19:14:38.955783 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.955766 2575 scope.go:117] "RemoveContainer" containerID="9c0f593d10ecc9d46bb6e21fb8f5708cc8c00b3c0f4865503df596a2f3e163e0" May 20 19:14:38.956046 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:14:38.956026 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c0f593d10ecc9d46bb6e21fb8f5708cc8c00b3c0f4865503df596a2f3e163e0\": container with ID starting with 9c0f593d10ecc9d46bb6e21fb8f5708cc8c00b3c0f4865503df596a2f3e163e0 not found: ID does not exist" containerID="9c0f593d10ecc9d46bb6e21fb8f5708cc8c00b3c0f4865503df596a2f3e163e0" May 20 19:14:38.956112 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.956058 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c0f593d10ecc9d46bb6e21fb8f5708cc8c00b3c0f4865503df596a2f3e163e0"} err="failed to get container status \"9c0f593d10ecc9d46bb6e21fb8f5708cc8c00b3c0f4865503df596a2f3e163e0\": rpc error: code = NotFound desc = could not find container \"9c0f593d10ecc9d46bb6e21fb8f5708cc8c00b3c0f4865503df596a2f3e163e0\": container with ID starting with 9c0f593d10ecc9d46bb6e21fb8f5708cc8c00b3c0f4865503df596a2f3e163e0 not found: ID does not exist" May 20 19:14:38.966852 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.966832 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-549f87f94d-vbq5k"] May 20 19:14:38.968474 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:38.968457 2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-549f87f94d-vbq5k"] May 20 19:14:39.000165 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:39.000139 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5754\" (UniqueName: \"kubernetes.io/projected/36afe091-344b-4e95-b97f-38d87396e820-kube-api-access-n5754\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:14:39.090219 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:39.090174 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36afe091-344b-4e95-b97f-38d87396e820" path="/var/lib/kubelet/pods/36afe091-344b-4e95-b97f-38d87396e820/volumes" May 20 19:14:39.700256 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:39.700221 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-549f87f94d-t5g2g"] May 20 19:14:39.700594 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:39.700549 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36afe091-344b-4e95-b97f-38d87396e820" containerName="manager" May 20 19:14:39.700594 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:39.700560 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="36afe091-344b-4e95-b97f-38d87396e820" containerName="manager" May 20 19:14:39.700663 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:39.700621 2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="36afe091-344b-4e95-b97f-38d87396e820" containerName="manager" May 20 19:14:39.704882 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:39.704860 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-549f87f94d-t5g2g" May 20 19:14:39.707484 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:39.707468 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zfwv9\"" May 20 19:14:39.712192 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:39.712136 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-549f87f94d-t5g2g"] May 20 19:14:39.812190 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:39.812153 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wndbw\" (UniqueName: \"kubernetes.io/projected/3f777817-88fb-469c-a7e4-88ff6375956a-kube-api-access-wndbw\") pod \"maas-controller-549f87f94d-t5g2g\" (UID: \"3f777817-88fb-469c-a7e4-88ff6375956a\") " pod="opendatahub/maas-controller-549f87f94d-t5g2g" May 20 19:14:39.913234 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:39.913170 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wndbw\" (UniqueName: \"kubernetes.io/projected/3f777817-88fb-469c-a7e4-88ff6375956a-kube-api-access-wndbw\") pod \"maas-controller-549f87f94d-t5g2g\" (UID: \"3f777817-88fb-469c-a7e4-88ff6375956a\") " pod="opendatahub/maas-controller-549f87f94d-t5g2g" May 20 19:14:39.922337 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:39.922298 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wndbw\" (UniqueName: \"kubernetes.io/projected/3f777817-88fb-469c-a7e4-88ff6375956a-kube-api-access-wndbw\") pod \"maas-controller-549f87f94d-t5g2g\" (UID: \"3f777817-88fb-469c-a7e4-88ff6375956a\") " pod="opendatahub/maas-controller-549f87f94d-t5g2g" May 20 19:14:40.015896 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:40.015805 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-549f87f94d-t5g2g" May 20 19:14:40.159892 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:40.143478 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-549f87f94d-t5g2g"] May 20 19:14:40.159892 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:14:40.145851 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f777817_88fb_469c_a7e4_88ff6375956a.slice/crio-7cd6d191a6fd4437514dd1f71ebb8bb003e128fb978c90c64bcf37eb4028972a WatchSource:0}: Error finding container 7cd6d191a6fd4437514dd1f71ebb8bb003e128fb978c90c64bcf37eb4028972a: Status 404 returned error can't find the container with id 7cd6d191a6fd4437514dd1f71ebb8bb003e128fb978c90c64bcf37eb4028972a May 20 19:14:40.956043 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:40.955950 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-549f87f94d-t5g2g" event={"ID":"3f777817-88fb-469c-a7e4-88ff6375956a","Type":"ContainerStarted","Data":"fafed055c90a8758ccfff130090ce290267d6384add13eed19fdbc49ccf656d6"} May 20 19:14:40.956043 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:40.955994 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-549f87f94d-t5g2g" event={"ID":"3f777817-88fb-469c-a7e4-88ff6375956a","Type":"ContainerStarted","Data":"7cd6d191a6fd4437514dd1f71ebb8bb003e128fb978c90c64bcf37eb4028972a"} May 20 19:14:40.956495 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:40.956052 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-549f87f94d-t5g2g" May 20 19:14:40.973739 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:40.973690 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-549f87f94d-t5g2g" podStartSLOduration=1.55723719 podStartE2EDuration="1.973675348s" podCreationTimestamp="2026-05-20 19:14:39 +0000 UTC" firstStartedPulling="2026-05-20 19:14:40.147069582 +0000 UTC m=+937.693028415" lastFinishedPulling="2026-05-20 19:14:40.563507728 +0000 UTC m=+938.109466573" observedRunningTime="2026-05-20 19:14:40.972664568 +0000 UTC m=+938.518623424" watchObservedRunningTime="2026-05-20 19:14:40.973675348 +0000 UTC m=+938.519634228" May 20 19:14:51.965862 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:14:51.965829 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-549f87f94d-t5g2g" May 20 19:19:03.057161 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:19:03.057131 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:19:03.058691 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:19:03.058674 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:24:03.081549 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:03.081518 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:24:03.083755 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:03.083734 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:24:48.051599 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:48.051558 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8"] May 20 19:24:48.052191 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:48.051932 2575 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" podUID="b53d672b-81cb-4d66-bc86-afaa95414437" containerName="manager" containerID="cri-o://32ce2d6756186d3a09d94574491fe40fce09e17943a8bf0cb6f4745a531759c7" gracePeriod=10 May 20 19:24:48.407994 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:48.407969 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" May 20 19:24:48.500368 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:48.500337 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqdzp\" (UniqueName: \"kubernetes.io/projected/b53d672b-81cb-4d66-bc86-afaa95414437-kube-api-access-nqdzp\") pod \"b53d672b-81cb-4d66-bc86-afaa95414437\" (UID: \"b53d672b-81cb-4d66-bc86-afaa95414437\") " May 20 19:24:48.500547 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:48.500429 2575 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b53d672b-81cb-4d66-bc86-afaa95414437-extensions-socket-volume\") pod \"b53d672b-81cb-4d66-bc86-afaa95414437\" (UID: \"b53d672b-81cb-4d66-bc86-afaa95414437\") " May 20 19:24:48.500887 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:48.500854 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b53d672b-81cb-4d66-bc86-afaa95414437-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b53d672b-81cb-4d66-bc86-afaa95414437" (UID: "b53d672b-81cb-4d66-bc86-afaa95414437"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 20 19:24:48.502352 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:48.502333 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b53d672b-81cb-4d66-bc86-afaa95414437-kube-api-access-nqdzp" (OuterVolumeSpecName: "kube-api-access-nqdzp") pod "b53d672b-81cb-4d66-bc86-afaa95414437" (UID: "b53d672b-81cb-4d66-bc86-afaa95414437"). InnerVolumeSpecName "kube-api-access-nqdzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 19:24:48.601310 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:48.601230 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqdzp\" (UniqueName: \"kubernetes.io/projected/b53d672b-81cb-4d66-bc86-afaa95414437-kube-api-access-nqdzp\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:24:48.601310 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:48.601259 2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b53d672b-81cb-4d66-bc86-afaa95414437-extensions-socket-volume\") on node \"ip-10-0-138-66.ec2.internal\" DevicePath \"\"" May 20 19:24:49.060006 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:49.059970 2575 generic.go:358] "Generic (PLEG): container finished" podID="b53d672b-81cb-4d66-bc86-afaa95414437" containerID="32ce2d6756186d3a09d94574491fe40fce09e17943a8bf0cb6f4745a531759c7" exitCode=0 May 20 19:24:49.060449 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:49.060031 2575 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" May 20 19:24:49.060449 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:49.060050 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" event={"ID":"b53d672b-81cb-4d66-bc86-afaa95414437","Type":"ContainerDied","Data":"32ce2d6756186d3a09d94574491fe40fce09e17943a8bf0cb6f4745a531759c7"} May 20 19:24:49.060449 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:49.060089 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8" event={"ID":"b53d672b-81cb-4d66-bc86-afaa95414437","Type":"ContainerDied","Data":"8b1b87537dbb6ac4c8d0d2f39db55ed8a9e1376ba2536759450ac11b40fa7e0d"} May 20 19:24:49.060449 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:49.060103 2575 scope.go:117] "RemoveContainer" containerID="32ce2d6756186d3a09d94574491fe40fce09e17943a8bf0cb6f4745a531759c7" May 20 19:24:49.068571 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:49.068554 2575 scope.go:117] "RemoveContainer" containerID="32ce2d6756186d3a09d94574491fe40fce09e17943a8bf0cb6f4745a531759c7" May 20 19:24:49.068787 ip-10-0-138-66 kubenswrapper[2575]: E0520 19:24:49.068770 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32ce2d6756186d3a09d94574491fe40fce09e17943a8bf0cb6f4745a531759c7\": container with ID starting with 32ce2d6756186d3a09d94574491fe40fce09e17943a8bf0cb6f4745a531759c7 not found: ID does not exist" containerID="32ce2d6756186d3a09d94574491fe40fce09e17943a8bf0cb6f4745a531759c7" May 20 19:24:49.068834 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:49.068795 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32ce2d6756186d3a09d94574491fe40fce09e17943a8bf0cb6f4745a531759c7"} err="failed to get container status \"32ce2d6756186d3a09d94574491fe40fce09e17943a8bf0cb6f4745a531759c7\": rpc error: code = NotFound desc = could not find container \"32ce2d6756186d3a09d94574491fe40fce09e17943a8bf0cb6f4745a531759c7\": container with ID starting with 32ce2d6756186d3a09d94574491fe40fce09e17943a8bf0cb6f4745a531759c7 not found: ID does not exist" May 20 19:24:49.081066 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:49.081035 2575 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8"] May 20 19:24:49.089155 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:49.089134 2575 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-pxtn8"] May 20 19:24:51.089833 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:24:51.089799 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b53d672b-81cb-4d66-bc86-afaa95414437" path="/var/lib/kubelet/pods/b53d672b-81cb-4d66-bc86-afaa95414437/volumes" May 20 19:25:54.145911 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.145830 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh"] May 20 19:25:54.148302 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.146226 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b53d672b-81cb-4d66-bc86-afaa95414437" containerName="manager" May 20 19:25:54.148302 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.146243 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b53d672b-81cb-4d66-bc86-afaa95414437" containerName="manager" May 20 19:25:54.148302 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.146308 2575 memory_manager.go:356] "RemoveStaleState removing state" podUID="b53d672b-81cb-4d66-bc86-afaa95414437" containerName="manager" May 20 19:25:54.149188 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.149173 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh" May 20 19:25:54.152334 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.152310 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sdqjb\"" May 20 19:25:54.160332 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.160311 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh"] May 20 19:25:54.284679 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.284645 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb326c2e-b800-415d-ab1f-068efd44f678-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-qfqvh\" (UID: \"bb326c2e-b800-415d-ab1f-068efd44f678\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh" May 20 19:25:54.284876 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.284783 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs29l\" (UniqueName: \"kubernetes.io/projected/bb326c2e-b800-415d-ab1f-068efd44f678-kube-api-access-fs29l\") pod \"kuadrant-operator-controller-manager-55c7f4c975-qfqvh\" (UID: \"bb326c2e-b800-415d-ab1f-068efd44f678\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh" May 20 19:25:54.386004 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.385948 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb326c2e-b800-415d-ab1f-068efd44f678-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-qfqvh\" (UID: \"bb326c2e-b800-415d-ab1f-068efd44f678\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh" May 20 19:25:54.386196 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.386074 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fs29l\" (UniqueName: \"kubernetes.io/projected/bb326c2e-b800-415d-ab1f-068efd44f678-kube-api-access-fs29l\") pod \"kuadrant-operator-controller-manager-55c7f4c975-qfqvh\" (UID: \"bb326c2e-b800-415d-ab1f-068efd44f678\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh" May 20 19:25:54.386402 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.386378 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb326c2e-b800-415d-ab1f-068efd44f678-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-qfqvh\" (UID: \"bb326c2e-b800-415d-ab1f-068efd44f678\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh" May 20 19:25:54.394767 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.394749 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs29l\" (UniqueName: \"kubernetes.io/projected/bb326c2e-b800-415d-ab1f-068efd44f678-kube-api-access-fs29l\") pod \"kuadrant-operator-controller-manager-55c7f4c975-qfqvh\" (UID: \"bb326c2e-b800-415d-ab1f-068efd44f678\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh" May 20 19:25:54.459460 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.459367 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh" May 20 19:25:54.594551 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.594430 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh"] May 20 19:25:54.597527 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:25:54.597497 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb326c2e_b800_415d_ab1f_068efd44f678.slice/crio-1906b6b828592f33674fe7543550d51da3170d8a77a3052533fd8ddfe7099de9 WatchSource:0}: Error finding container 1906b6b828592f33674fe7543550d51da3170d8a77a3052533fd8ddfe7099de9: Status 404 returned error can't find the container with id 1906b6b828592f33674fe7543550d51da3170d8a77a3052533fd8ddfe7099de9 May 20 19:25:54.599570 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:54.599552 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 20 19:25:55.281363 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:55.281326 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh" event={"ID":"bb326c2e-b800-415d-ab1f-068efd44f678","Type":"ContainerStarted","Data":"e52f987610b7d8f1d15c5dcdffb05bf53bcd353691270aad58f9b6a7921a1408"} May 20 19:25:55.281363 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:55.281365 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh" event={"ID":"bb326c2e-b800-415d-ab1f-068efd44f678","Type":"ContainerStarted","Data":"1906b6b828592f33674fe7543550d51da3170d8a77a3052533fd8ddfe7099de9"} May 20 19:25:55.281778 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:55.281450 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh" May 20 19:25:55.301685 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:25:55.301634 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh" podStartSLOduration=1.3016195 podStartE2EDuration="1.3016195s" podCreationTimestamp="2026-05-20 19:25:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 19:25:55.300256336 +0000 UTC m=+1612.846215192" watchObservedRunningTime="2026-05-20 19:25:55.3016195 +0000 UTC m=+1612.847578354" May 20 19:26:06.286751 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:26:06.286715 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-qfqvh" May 20 19:29:03.105287 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:29:03.105256 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:29:03.109373 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:29:03.109352 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:34:03.133262 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:34:03.133230 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:34:03.139119 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:34:03.139095 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:35:40.145685 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:40.145641 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-6lwqb_4520054d-c70f-41fe-8011-220449c622b3/manager/0.log" May 20 19:35:40.265536 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:40.265506 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5d4c4dbf77-prb6g_537eb9cc-53ef-4363-9fb7-20e0cfc7edbc/maas-api/0.log" May 20 19:35:40.589638 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:40.589608 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-549f87f94d-t5g2g_3f777817-88fb-469c-a7e4-88ff6375956a/manager/0.log" May 20 19:35:40.702157 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:40.702129 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-qlgtq_13823e31-0602-45a5-bb05-e980ef446c1c/manager/2.log" May 20 19:35:40.953930 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:40.953840 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6764c5ff7d-g9jks_8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2/manager/0.log" May 20 19:35:41.166768 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:41.166738 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-kx5fw_7430e17b-f4a9-44c4-8922-9e4fd7510d94/postgres/0.log" May 20 19:35:42.667468 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:42.667442 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-bc9kf_92c15793-b2d1-402e-bb21-766b22dd60dd/manager/0.log" May 20 19:35:43.016808 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:43.016734 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-qfqvh_bb326c2e-b800-415d-ab1f-068efd44f678/manager/0.log" May 20 19:35:43.714439 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:43.714414 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-gcf2b_fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0/discovery/0.log" May 20 19:35:43.936221 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:43.936184 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8958bfff9-757xw_a227a7b3-34e3-4f03-bf21-67e0de1c547e/kube-auth-proxy/0.log" May 20 19:35:44.175499 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:44.175471 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-jggd8_40dfee4e-ef27-4500-b206-984941dcb97f/payload-processing/0.log" May 20 19:35:44.647699 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:44.647672 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp_a48aab9b-0846-41f5-b6ec-4664d7375ec8/storage-initializer/0.log" May 20 19:35:44.664054 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:44.664027 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-97pcp_a48aab9b-0846-41f5-b6ec-4664d7375ec8/main/0.log" May 20 19:35:44.779550 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:44.779523 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4_bd786bb3-fcbc-41da-a19d-b21a0836c109/storage-initializer/0.log" May 20 19:35:44.792910 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:44.792880 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-ck2p4_bd786bb3-fcbc-41da-a19d-b21a0836c109/main/0.log" May 20 19:35:44.903583 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:44.903512 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-p579g_9b041ef4-8ccb-488a-9f16-2a7bcf072cc2/storage-initializer/0.log" May 20 19:35:44.916046 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:44.916026 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-p579g_9b041ef4-8ccb-488a-9f16-2a7bcf072cc2/main/0.log" May 20 19:35:45.168634 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:45.168561 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7_300abe09-50b4-40c9-9764-8883f3fb703a/storage-initializer/0.log" May 20 19:35:45.180896 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:45.180874 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-5whd7_300abe09-50b4-40c9-9764-8883f3fb703a/main/0.log" May 20 19:35:52.652470 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:52.652437 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j9r94_1b20dd7a-c2dd-489e-bd5e-28916190af81/global-pull-secret-syncer/0.log" May 20 19:35:52.791321 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:52.791293 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-klzxc_e8bdc28d-e9ef-4ce6-9a83-1947176e82a4/konnectivity-agent/0.log" May 20 19:35:52.892313 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:52.892283 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-66.ec2.internal_9d991159dd187170fe148f8050e8b485/haproxy/0.log" May 20 19:35:57.308550 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:57.308521 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-bc9kf_92c15793-b2d1-402e-bb21-766b22dd60dd/manager/0.log" May 20 19:35:57.455674 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:57.455643 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-qfqvh_bb326c2e-b800-415d-ab1f-068efd44f678/manager/0.log" May 20 19:35:59.509635 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:59.509605 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n56hr_5f2ade48-887b-4500-8cbc-2ce26b69f1a7/node-exporter/0.log" May 20 19:35:59.530652 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:59.530626 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n56hr_5f2ade48-887b-4500-8cbc-2ce26b69f1a7/kube-rbac-proxy/0.log" May 20 19:35:59.550484 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:59.550459 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n56hr_5f2ade48-887b-4500-8cbc-2ce26b69f1a7/init-textfile/0.log" May 20 19:35:59.933402 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:59.933357 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6895ffb49b-75r8q_67cfd3a3-45b9-49a3-9252-d1dbb354b137/telemeter-client/0.log" May 20 19:35:59.954785 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:59.954757 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6895ffb49b-75r8q_67cfd3a3-45b9-49a3-9252-d1dbb354b137/reload/0.log" May 20 19:35:59.977636 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:35:59.977613 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6895ffb49b-75r8q_67cfd3a3-45b9-49a3-9252-d1dbb354b137/kube-rbac-proxy/0.log" May 20 19:36:01.405233 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.405172 2575 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh"] May 20 19:36:01.408986 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.408962 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.411525 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.411505 2575 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6lhgs\"/\"default-dockercfg-tmvzm\"" May 20 19:36:01.411525 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.411514 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6lhgs\"/\"openshift-service-ca.crt\"" May 20 19:36:01.412727 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.412697 2575 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6lhgs\"/\"kube-root-ca.crt\"" May 20 19:36:01.418336 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.418316 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh"] May 20 19:36:01.457224 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.457185 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c39f4a98-714b-480f-834e-b64ded0ec8b5-lib-modules\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.457400 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.457286 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jpv4\" (UniqueName: \"kubernetes.io/projected/c39f4a98-714b-480f-834e-b64ded0ec8b5-kube-api-access-2jpv4\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.457400 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.457349 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c39f4a98-714b-480f-834e-b64ded0ec8b5-podres\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.457482 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.457398 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c39f4a98-714b-480f-834e-b64ded0ec8b5-sys\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.457482 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.457415 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c39f4a98-714b-480f-834e-b64ded0ec8b5-proc\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.557837 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.557799 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2jpv4\" (UniqueName: \"kubernetes.io/projected/c39f4a98-714b-480f-834e-b64ded0ec8b5-kube-api-access-2jpv4\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.558006 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.557847 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c39f4a98-714b-480f-834e-b64ded0ec8b5-podres\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.558006 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.557949 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c39f4a98-714b-480f-834e-b64ded0ec8b5-podres\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.558075 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.558016 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c39f4a98-714b-480f-834e-b64ded0ec8b5-sys\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.558075 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.558049 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c39f4a98-714b-480f-834e-b64ded0ec8b5-proc\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.558166 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.558103 2575 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c39f4a98-714b-480f-834e-b64ded0ec8b5-lib-modules\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.558166 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.558124 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c39f4a98-714b-480f-834e-b64ded0ec8b5-sys\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.558307 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.558194 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c39f4a98-714b-480f-834e-b64ded0ec8b5-proc\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.558307 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.558233 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c39f4a98-714b-480f-834e-b64ded0ec8b5-lib-modules\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.565868 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.565840 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jpv4\" (UniqueName: \"kubernetes.io/projected/c39f4a98-714b-480f-834e-b64ded0ec8b5-kube-api-access-2jpv4\") pod \"perf-node-gather-daemonset-g5hvh\" (UID: \"c39f4a98-714b-480f-834e-b64ded0ec8b5\") " pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.719815 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.719726 2575 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:01.854348 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.854293 2575 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh"] May 20 19:36:01.856831 ip-10-0-138-66 kubenswrapper[2575]: W0520 19:36:01.856801 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc39f4a98_714b_480f_834e_b64ded0ec8b5.slice/crio-f16d059da1191ff56ec0b61fcd05026cd656f46e1a272f690047d6980098b2e1 WatchSource:0}: Error finding container f16d059da1191ff56ec0b61fcd05026cd656f46e1a272f690047d6980098b2e1: Status 404 returned error can't find the container with id f16d059da1191ff56ec0b61fcd05026cd656f46e1a272f690047d6980098b2e1 May 20 19:36:01.858379 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:01.858360 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 20 19:36:02.403049 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:02.403015 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" event={"ID":"c39f4a98-714b-480f-834e-b64ded0ec8b5","Type":"ContainerStarted","Data":"0e7dbc090a6d1a0ea4c165dc53dd1ff26af0659e32181a7c364893f6b69d3307"} May 20 19:36:02.403049 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:02.403051 2575 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" event={"ID":"c39f4a98-714b-480f-834e-b64ded0ec8b5","Type":"ContainerStarted","Data":"f16d059da1191ff56ec0b61fcd05026cd656f46e1a272f690047d6980098b2e1"} May 20 19:36:02.403288 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:02.403140 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:02.418914 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:02.418852 2575 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" podStartSLOduration=1.418836622 podStartE2EDuration="1.418836622s" podCreationTimestamp="2026-05-20 19:36:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 19:36:02.418508106 +0000 UTC m=+2219.964466961" watchObservedRunningTime="2026-05-20 19:36:02.418836622 +0000 UTC m=+2219.964795516" May 20 19:36:03.424846 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:03.424815 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-95f7s_d601dbc0-665a-4564-833e-bef2c554536c/dns/0.log" May 20 19:36:03.443941 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:03.443913 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-95f7s_d601dbc0-665a-4564-833e-bef2c554536c/kube-rbac-proxy/0.log" May 20 19:36:03.580235 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:03.580183 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gpvmh_23268351-bee9-4a36-937b-5d3eccdf2806/dns-node-resolver/0.log" May 20 19:36:04.059526 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:04.059496 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-66dc6fb849-rxgv6_9f802bde-bca5-434f-b866-7e5cc08054b9/registry/0.log" May 20 19:36:04.079012 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:04.078988 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d5z52_f2e38623-90a4-414d-9517-087b938f83c1/node-ca/0.log" May 20 19:36:04.991689 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:04.991642 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-gcf2b_fb1fea04-5a17-4fc8-9ae7-79d5a27f47c0/discovery/0.log" May 20 19:36:05.047587 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:05.047560 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8958bfff9-757xw_a227a7b3-34e3-4f03-bf21-67e0de1c547e/kube-auth-proxy/0.log" May 20 19:36:05.153120 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:05.153086 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-jggd8_40dfee4e-ef27-4500-b206-984941dcb97f/payload-processing/0.log" May 20 19:36:05.700216 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:05.700184 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lhst4_b4c6b123-27fc-45c3-a22f-6cf99088f182/serve-healthcheck-canary/0.log" May 20 19:36:06.168722 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:06.168695 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5kdhc_e31f8582-cd37-4c6f-a313-cd02ca18d675/kube-rbac-proxy/0.log" May 20 19:36:06.193618 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:06.193592 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5kdhc_e31f8582-cd37-4c6f-a313-cd02ca18d675/exporter/0.log" May 20 19:36:06.213113 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:06.213081 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5kdhc_e31f8582-cd37-4c6f-a313-cd02ca18d675/extractor/0.log" May 20 19:36:08.190809 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:08.190772 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-6lwqb_4520054d-c70f-41fe-8011-220449c622b3/manager/0.log" May 20 19:36:08.220484 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:08.220454 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5d4c4dbf77-prb6g_537eb9cc-53ef-4363-9fb7-20e0cfc7edbc/maas-api/0.log" May 20 19:36:08.330010 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:08.329974 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-549f87f94d-t5g2g_3f777817-88fb-469c-a7e4-88ff6375956a/manager/0.log" May 20 19:36:08.348995 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:08.348965 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-qlgtq_13823e31-0602-45a5-bb05-e980ef446c1c/manager/1.log" May 20 19:36:08.361484 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:08.361453 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-qlgtq_13823e31-0602-45a5-bb05-e980ef446c1c/manager/2.log" May 20 19:36:08.416544 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:08.416516 2575 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6lhgs/perf-node-gather-daemonset-g5hvh" May 20 19:36:08.438739 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:08.438709 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6764c5ff7d-g9jks_8ecf7c0c-4e13-4eaf-a5f5-85f05a7a10d2/manager/0.log" May 20 19:36:08.489948 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:08.489866 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-kx5fw_7430e17b-f4a9-44c4-8922-9e4fd7510d94/postgres/0.log" May 20 19:36:15.239361 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:15.239328 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7fszs_1c634114-53bc-40fb-abff-081af0ff74eb/kube-multus-additional-cni-plugins/0.log" May 20 19:36:15.263971 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:15.263943 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7fszs_1c634114-53bc-40fb-abff-081af0ff74eb/egress-router-binary-copy/0.log" May 20 19:36:15.282221 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:15.282185 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7fszs_1c634114-53bc-40fb-abff-081af0ff74eb/cni-plugins/0.log" May 20 19:36:15.301301 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:15.301276 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7fszs_1c634114-53bc-40fb-abff-081af0ff74eb/bond-cni-plugin/0.log" May 20 19:36:15.320339 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:15.320310 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7fszs_1c634114-53bc-40fb-abff-081af0ff74eb/routeoverride-cni/0.log" May 20 19:36:15.340103 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:15.340082 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7fszs_1c634114-53bc-40fb-abff-081af0ff74eb/whereabouts-cni-bincopy/0.log" May 20 19:36:15.364535 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:15.364505 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7fszs_1c634114-53bc-40fb-abff-081af0ff74eb/whereabouts-cni/0.log" May 20 19:36:15.760684 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:15.760643 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xlhks_a6c9126b-be8e-4c5f-b67b-940897245170/kube-multus/0.log" May 20 19:36:15.821297 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:15.821268 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-84tr8_8c283ba7-2436-4f29-b86c-97340c785bee/network-metrics-daemon/0.log" May 20 19:36:15.838426 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:15.838402 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-84tr8_8c283ba7-2436-4f29-b86c-97340c785bee/kube-rbac-proxy/0.log" May 20 19:36:17.292111 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:17.292082 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-controller/0.log" May 20 19:36:17.309382 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:17.309361 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/0.log" May 20 19:36:17.318518 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:17.318490 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovn-acl-logging/1.log" May 20 19:36:17.336750 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:17.336724 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/kube-rbac-proxy-node/0.log" May 20 19:36:17.357649 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:17.357620 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/kube-rbac-proxy-ovn-metrics/0.log" May 20 19:36:17.374791 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:17.374760 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/northd/0.log" May 20 19:36:17.393508 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:17.393479 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/nbdb/0.log" May 20 19:36:17.412696 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:17.412672 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/sbdb/0.log" May 20 19:36:17.510562 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:17.510526 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-szh8j_b9b783e1-b420-4311-816e-61b4df8bb617/ovnkube-controller/0.log" May 20 19:36:18.640842 ip-10-0-138-66 kubenswrapper[2575]: I0520 19:36:18.640770 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-8hwbz_44861413-0803-43a6-9a94-e2e0085120c6/network-check-target-container/0.log"