Jun 03 13:53:43.935661 ip-10-0-134-89 systemd[1]: Starting Kubernetes Kubelet... Jun 03 13:53:44.385474 ip-10-0-134-89 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 13:53:44.385474 ip-10-0-134-89 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 03 13:53:44.385474 ip-10-0-134-89 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 13:53:44.385474 ip-10-0-134-89 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 03 13:53:44.385474 ip-10-0-134-89 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 13:53:44.387471 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.387371 2565 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 03 13:53:44.391345 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391328 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 13:53:44.391345 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391345 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391349 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391352 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391355 2565 feature_gate.go:328] unrecognized feature gate: Example Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391358 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391361 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391376 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391380 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391383 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391386 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391388 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391391 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391394 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391397 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391399 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391402 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391404 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391413 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391416 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 13:53:44.391422 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391418 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391423 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391427 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391430 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391433 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391436 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391439 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391442 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391450 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391452 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391455 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391458 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391460 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391463 2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391465 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391469 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391471 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391474 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391477 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 13:53:44.391926 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391479 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391482 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391485 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391488 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391490 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391493 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391496 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391498 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391501 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391503 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391506 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391508 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391511 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391514 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391517 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391521 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391524 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391527 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391530 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391533 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 13:53:44.392432 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391535 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391538 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391541 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391544 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391548 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391552 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391554 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391557 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391559 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391562 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391565 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391567 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391570 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391573 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391576 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391580 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391583 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391586 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391589 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391591 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 13:53:44.392933 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391594 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391596 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391599 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391601 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391604 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391607 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.391610 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392032 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392037 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392040 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392043 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392046 2565 feature_gate.go:328] unrecognized feature gate: Example Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392048 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392051 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392054 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392057 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392059 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392064 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392068 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 13:53:44.393445 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392071 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392074 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392077 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392080 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392082 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392085 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392088 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392090 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392093 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392096 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392099 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392101 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392104 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392106 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392109 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392111 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392114 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392116 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392119 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392122 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 13:53:44.393912 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392127 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392130 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392132 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392135 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392137 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392140 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392143 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392145 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392148 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392150 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392153 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392155 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392158 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392161 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392164 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392166 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392169 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392172 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392174 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392177 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 13:53:44.394477 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392179 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392182 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392186 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392188 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392191 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392194 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392196 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392199 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392201 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392204 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392206 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392209 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392213 2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392216 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392218 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392221 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392224 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392226 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392229 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392232 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 13:53:44.394977 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392234 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392237 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392239 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392242 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392245 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392248 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392250 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392253 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392256 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392259 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392263 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392266 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392269 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392271 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392346 2565 flags.go:64] FLAG: --address="0.0.0.0" Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392354 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392361 2565 flags.go:64] FLAG: --anonymous-auth="true" Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392378 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392383 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392386 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 03 13:53:44.395500 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392390 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392395 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392398 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392401 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392405 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392409 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392412 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392415 2565 flags.go:64] FLAG: --cgroup-root="" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392418 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392421 2565 flags.go:64] FLAG: --client-ca-file="" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392424 2565 flags.go:64] FLAG: --cloud-config="" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392427 2565 flags.go:64] FLAG: --cloud-provider="external" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392430 2565 flags.go:64] FLAG: --cluster-dns="[]" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392435 2565 flags.go:64] FLAG: --cluster-domain="" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392438 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392441 2565 flags.go:64] FLAG: --config-dir="" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392444 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392448 2565 flags.go:64] FLAG: --container-log-max-files="5" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392452 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392455 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392458 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392462 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392465 2565 flags.go:64] FLAG: --contention-profiling="false" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392468 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 03 13:53:44.396006 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392471 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392475 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392477 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392482 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392486 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392490 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392493 2565 flags.go:64] FLAG: --enable-load-reader="false" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392496 2565 flags.go:64] FLAG: --enable-server="true" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392499 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392504 2565 flags.go:64] FLAG: --event-burst="100" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392507 2565 flags.go:64] FLAG: --event-qps="50" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392510 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392514 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392517 2565 flags.go:64] FLAG: --eviction-hard="" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392521 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392524 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392527 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392530 2565 flags.go:64] FLAG: --eviction-soft="" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392533 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392537 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392540 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392543 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392546 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392549 2565 flags.go:64] FLAG: --fail-swap-on="true" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392552 2565 flags.go:64] FLAG: --feature-gates="" Jun 03 13:53:44.396616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392556 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392559 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392562 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392566 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392569 2565 flags.go:64] FLAG: --healthz-port="10248" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392572 2565 flags.go:64] FLAG: --help="false" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392575 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392578 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392581 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392584 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392587 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392591 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392594 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392597 2565 flags.go:64] FLAG: --image-service-endpoint="" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392600 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392606 2565 flags.go:64] FLAG: --kube-api-burst="100" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392610 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392613 2565 flags.go:64] FLAG: --kube-api-qps="50" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392616 2565 flags.go:64] FLAG: --kube-reserved="" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392620 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392623 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392626 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392629 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392632 2565 flags.go:64] FLAG: --lock-file="" Jun 03 13:53:44.397292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392636 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392639 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392642 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392647 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392650 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392653 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392657 2565 flags.go:64] FLAG: --logging-format="text" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392660 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392663 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392666 2565 flags.go:64] FLAG: --manifest-url="" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392669 2565 flags.go:64] FLAG: --manifest-url-header="" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392673 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392676 2565 flags.go:64] FLAG: --max-open-files="1000000" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392681 2565 flags.go:64] FLAG: --max-pods="110" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392684 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392687 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392690 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392693 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392696 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392699 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392702 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392710 2565 flags.go:64] FLAG: --node-status-max-images="50" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392714 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392718 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jun 03 13:53:44.397915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392721 2565 flags.go:64] FLAG: --pod-cidr="" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392724 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392730 2565 flags.go:64] FLAG: --pod-manifest-path="" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392734 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392737 2565 flags.go:64] FLAG: --pods-per-core="0" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392740 2565 flags.go:64] FLAG: --port="10250" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392743 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392746 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ced329508e47fd89" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392749 2565 flags.go:64] FLAG: --qos-reserved="" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392752 2565 flags.go:64] FLAG: --read-only-port="10255" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392755 2565 flags.go:64] FLAG: --register-node="true" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392758 2565 flags.go:64] FLAG: --register-schedulable="true" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392762 2565 flags.go:64] FLAG: --register-with-taints="" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392765 2565 flags.go:64] FLAG: --registry-burst="10" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392768 2565 flags.go:64] FLAG: --registry-qps="5" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392771 2565 flags.go:64] FLAG: --reserved-cpus="" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392774 2565 flags.go:64] FLAG: --reserved-memory="" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392778 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392781 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392784 2565 flags.go:64] FLAG: --rotate-certificates="false" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392787 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392790 2565 flags.go:64] FLAG: --runonce="false" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392793 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392796 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392799 2565 flags.go:64] FLAG: --seccomp-default="false" Jun 03 13:53:44.398523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392802 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392805 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392808 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392811 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392814 2565 flags.go:64] FLAG: --storage-driver-password="root" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392818 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392821 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392825 2565 flags.go:64] FLAG: --storage-driver-user="root" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392828 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392831 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392834 2565 flags.go:64] FLAG: --system-cgroups="" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392838 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392843 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392846 2565 flags.go:64] FLAG: --tls-cert-file="" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392849 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392853 2565 flags.go:64] FLAG: --tls-min-version="" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392856 2565 flags.go:64] FLAG: --tls-private-key-file="" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392858 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392861 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392864 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392867 2565 flags.go:64] FLAG: --v="2" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392872 2565 flags.go:64] FLAG: --version="false" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392877 2565 flags.go:64] FLAG: --vmodule="" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392881 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.392884 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 03 13:53:44.399168 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392994 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.392998 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393002 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393005 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393008 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393011 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393014 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393017 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393020 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393023 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393025 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393028 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393031 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393034 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393038 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393041 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393044 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393046 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393050 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393055 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 13:53:44.399842 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393058 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393060 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393063 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393074 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393077 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393080 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393083 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393085 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393088 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393091 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393094 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393096 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393099 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393102 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393105 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393107 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393110 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393113 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393116 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 13:53:44.400484 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393119 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393122 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393125 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393127 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393130 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393133 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393136 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393140 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393143 2565 feature_gate.go:328] unrecognized feature gate: Example Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393146 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393149 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393151 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393154 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393156 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393159 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393162 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393165 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393167 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393170 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393172 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 13:53:44.400991 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393175 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393177 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393180 2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393183 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393186 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393188 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393191 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393196 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393198 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393201 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393204 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393206 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393209 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393211 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393214 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393217 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393221 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393224 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393227 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393232 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 13:53:44.401579 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393236 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393239 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393242 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393245 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393247 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393250 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.393253 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.393799 2565 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.400284 2565 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.400300 2565 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400355 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400360 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400377 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400382 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400385 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 13:53:44.402089 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400389 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400392 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400394 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400397 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400400 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400403 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400406 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400408 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400411 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400413 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400416 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400419 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400422 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400425 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400428 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400430 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400433 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400437 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400440 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400443 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 13:53:44.402501 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400445 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400448 2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400451 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400453 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400456 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400460 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400464 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400468 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400471 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400474 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400476 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400479 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400482 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400485 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400487 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400490 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400493 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400496 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400499 2565 feature_gate.go:328] unrecognized feature gate: Example Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400502 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 13:53:44.403005 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400505 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400509 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400513 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400516 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400519 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400522 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400525 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400528 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400531 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400533 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400537 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400540 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400543 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400545 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400548 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400551 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400554 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400557 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400561 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 13:53:44.403522 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400563 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400566 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400569 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400571 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400574 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400577 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400580 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400583 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400585 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400588 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400591 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400594 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400596 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400599 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400601 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400604 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400607 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400609 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400612 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400614 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 13:53:44.404003 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400617 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400620 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.400625 2565 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400729 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400734 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400737 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400740 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400743 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400746 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400748 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400751 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400753 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400756 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400759 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400762 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400765 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 13:53:44.404517 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400767 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400770 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400773 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400776 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400779 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400782 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400785 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400788 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400792 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400795 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400798 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400801 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400804 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400806 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400809 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400811 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400814 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400818 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400820 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400824 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 13:53:44.404929 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400827 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400830 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400833 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400836 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400838 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400841 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400844 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400846 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400849 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400852 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400855 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400858 2565 feature_gate.go:328] unrecognized feature gate: Example Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400860 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400863 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400866 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400868 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400871 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400873 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400876 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400879 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 13:53:44.405525 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400882 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400886 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400889 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400892 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400894 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400897 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400900 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400902 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400905 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400908 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400910 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400913 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400916 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400918 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400922 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400925 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400927 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400930 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400932 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 13:53:44.406036 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400935 2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400938 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400940 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400943 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400945 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400948 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400950 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400953 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400955 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400958 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400961 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400963 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400966 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:44.400968 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.400973 2565 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 13:53:44.406521 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.401809 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jun 03 13:53:44.406896 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.403649 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 03 13:53:44.406896 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.404583 2565 server.go:1019] "Starting client certificate rotation" Jun 03 13:53:44.406896 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.404680 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 13:53:44.406896 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.404716 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 13:53:44.429292 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.429272 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 13:53:44.431052 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.431031 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 13:53:44.448795 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.448770 2565 log.go:25] "Validated CRI v1 runtime API" Jun 03 13:53:44.455814 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.455793 2565 log.go:25] "Validated CRI v1 image API" Jun 03 13:53:44.458045 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.458026 2565 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 03 13:53:44.459567 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.459552 2565 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 13:53:44.460242 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.460224 2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 da31d86a-7845-4196-a749-efd358201c81:/dev/nvme0n1p4 ed1fcbfa-d6c9-4015-bba6-15da21ad34b7:/dev/nvme0n1p3] Jun 03 13:53:44.460281 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.460242 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 03 13:53:44.464567 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.464460 2565 manager.go:217] Machine: {Timestamp:2026-06-03 13:53:44.463925046 +0000 UTC m=+0.408568992 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101007 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec233712eab3c7eef3e78611acdc2be1 SystemUUID:ec233712-eab3-c7ee-f3e7-8611acdc2be1 BootID:230e34bc-364d-4680-8f1a-266b0f9a318d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c2:1a:5d:78:c9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c2:1a:5d:78:c9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:46:2e:ef:b3:87 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 03 13:53:44.465773 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.465762 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 03 13:53:44.465864 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.465852 2565 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 03 13:53:44.467395 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.467366 2565 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 03 13:53:44.467532 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.467398 2565 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-89.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 03 13:53:44.467577 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.467542 2565 topology_manager.go:138] "Creating topology manager with none policy" Jun 03 13:53:44.467577 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.467551 2565 container_manager_linux.go:306] "Creating device plugin manager" Jun 03 13:53:44.467577 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.467559 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 13:53:44.468297 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.468287 2565 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 13:53:44.469027 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.469016 2565 state_mem.go:36] "Initialized new in-memory state store" Jun 03 13:53:44.469148 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.469139 2565 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 03 13:53:44.471627 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.471617 2565 kubelet.go:491] "Attempting to sync node with API server" Jun 03 13:53:44.471664 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.471632 2565 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 03 13:53:44.471664 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.471644 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 03 13:53:44.471664 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.471653 2565 kubelet.go:397] "Adding apiserver pod source" Jun 03 13:53:44.471664 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.471662 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 03 13:53:44.472680 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.472664 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 13:53:44.472781 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.472687 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 13:53:44.475623 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.475606 2565 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 03 13:53:44.477379 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.477363 2565 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 03 13:53:44.479651 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.479634 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 03 13:53:44.479713 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.479657 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 03 13:53:44.479713 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.479663 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 03 13:53:44.479713 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.479669 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 03 13:53:44.479713 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.479678 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 03 13:53:44.479713 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.479685 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 03 13:53:44.479713 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.479690 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 03 13:53:44.479713 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.479696 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 03 13:53:44.479713 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.479703 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 03 13:53:44.479713 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.479708 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 03 13:53:44.479713 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.479717 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 03 13:53:44.479971 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.479723 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 03 13:53:44.480417 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.480407 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 03 13:53:44.480417 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.480417 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 03 13:53:44.482749 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.482730 2565 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-89.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 03 13:53:44.482944 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.482923 2565 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 03 13:53:44.483825 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.483812 2565 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 03 13:53:44.483878 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.483850 2565 server.go:1295] "Started kubelet" Jun 03 13:53:44.483991 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.483955 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 03 13:53:44.483991 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.483958 2565 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 03 13:53:44.484084 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.484022 2565 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 03 13:53:44.486836 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.486760 2565 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 03 13:53:44.487444 ip-10-0-134-89 systemd[1]: Started Kubernetes Kubelet. Jun 03 13:53:44.489983 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.489467 2565 server.go:317] "Adding debug handlers to kubelet server" Jun 03 13:53:44.493782 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.493764 2565 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-89.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 03 13:53:44.493865 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.493845 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 03 13:53:44.494180 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.494164 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 03 13:53:44.494392 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.494371 2565 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 03 13:53:44.494392 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.494388 2565 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 03 13:53:44.496184 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.496164 2565 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 03 13:53:44.496293 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.496169 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-89.ec2.internal\" not found" Jun 03 13:53:44.496293 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.496280 2565 reconstruct.go:97] "Volume reconstruction finished" Jun 03 13:53:44.496293 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.496289 2565 reconciler.go:26] "Reconciler: start to sync state" Jun 03 13:53:44.496861 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.496843 2565 factory.go:55] Registering systemd factory Jun 03 13:53:44.496861 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.496864 2565 factory.go:223] Registration of the systemd container factory successfully Jun 03 13:53:44.497094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.497082 2565 factory.go:153] Registering CRI-O factory Jun 03 13:53:44.497094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.497096 2565 factory.go:223] Registration of the crio container factory successfully Jun 03 13:53:44.497206 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.497144 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 03 13:53:44.497206 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.497162 2565 factory.go:103] Registering Raw factory Jun 03 13:53:44.497206 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.497190 2565 manager.go:1196] Started watching for new ooms in manager Jun 03 13:53:44.497692 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.497680 2565 manager.go:319] Starting recovery of all containers Jun 03 13:53:44.498203 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.498174 2565 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 03 13:53:44.504846 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.504820 2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-89.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 03 13:53:44.504937 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.504914 2565 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 03 13:53:44.505981 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.504996 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-89.ec2.internal.18b5969c01c7a37b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-89.ec2.internal,UID:ip-10-0-134-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-89.ec2.internal,},FirstTimestamp:2026-06-03 13:53:44.483824507 +0000 UTC m=+0.428468452,LastTimestamp:2026-06-03 13:53:44.483824507 +0000 UTC m=+0.428468452,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-89.ec2.internal,}" Jun 03 13:53:44.507822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.507806 2565 manager.go:324] Recovery completed Jun 03 13:53:44.512004 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.511992 2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 13:53:44.517744 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.517728 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasSufficientMemory" Jun 03 13:53:44.517810 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.517758 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 13:53:44.517810 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.517768 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasSufficientPID" Jun 03 13:53:44.518717 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.518704 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 03 13:53:44.518717 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.518717 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 03 13:53:44.518829 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.518735 2565 state_mem.go:36] "Initialized new in-memory state store" Jun 03 13:53:44.520624 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.520612 2565 policy_none.go:49] "None policy: Start" Jun 03 13:53:44.520665 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.520629 2565 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 03 13:53:44.520665 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.520639 2565 state_mem.go:35] "Initializing new in-memory state store" Jun 03 13:53:44.524306 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.524290 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hb8rh" Jun 03 13:53:44.531171 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.531155 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hb8rh" Jun 03 13:53:44.532952 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.532889 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-89.ec2.internal.18b5969c03cd379b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-89.ec2.internal,UID:ip-10-0-134-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-89.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-89.ec2.internal,},FirstTimestamp:2026-06-03 13:53:44.517744539 +0000 UTC m=+0.462388484,LastTimestamp:2026-06-03 13:53:44.517744539 +0000 UTC m=+0.462388484,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-89.ec2.internal,}" Jun 03 13:53:44.555204 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.554553 2565 manager.go:341] "Starting Device Plugin manager" Jun 03 13:53:44.555204 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.554586 2565 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 03 13:53:44.555204 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.554595 2565 server.go:85] "Starting device plugin registration server" Jun 03 13:53:44.555204 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.554831 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 03 13:53:44.555204 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.554842 2565 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 03 13:53:44.555204 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.554928 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 03 13:53:44.555204 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.555023 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 03 13:53:44.555204 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.555035 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 03 13:53:44.555602 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.555563 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 03 13:53:44.555646 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.555603 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-89.ec2.internal\" not found" Jun 03 13:53:44.617256 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.617216 2565 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 03 13:53:44.618385 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.618365 2565 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 03 13:53:44.618459 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.618390 2565 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 03 13:53:44.618459 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.618409 2565 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 03 13:53:44.618459 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.618415 2565 kubelet.go:2451] "Starting kubelet main sync loop" Jun 03 13:53:44.618459 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.618450 2565 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 03 13:53:44.625554 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.625532 2565 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 13:53:44.655452 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.655393 2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 13:53:44.656450 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.656435 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasSufficientMemory" Jun 03 13:53:44.656516 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.656465 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 13:53:44.656516 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.656479 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasSufficientPID" Jun 03 13:53:44.656516 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.656511 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.664764 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.664750 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.664807 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.664772 2565 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-89.ec2.internal\": node \"ip-10-0-134-89.ec2.internal\" not found" Jun 03 13:53:44.682284 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.682263 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-89.ec2.internal\" not found" Jun 03 13:53:44.718731 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.718709 2565 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-89.ec2.internal"] Jun 03 13:53:44.718818 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.718782 2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 13:53:44.719632 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.719618 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasSufficientMemory" Jun 03 13:53:44.719701 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.719644 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 13:53:44.719701 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.719653 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasSufficientPID" Jun 03 13:53:44.720728 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.720717 2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 13:53:44.720901 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.720887 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.720954 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.720920 2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 13:53:44.721396 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.721367 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasSufficientMemory" Jun 03 13:53:44.721396 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.721398 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 13:53:44.721529 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.721410 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasSufficientPID" Jun 03 13:53:44.721529 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.721367 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasSufficientMemory" Jun 03 13:53:44.721529 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.721473 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 13:53:44.721529 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.721486 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasSufficientPID" Jun 03 13:53:44.722464 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.722452 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.722511 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.722480 2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 13:53:44.723077 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.723063 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasSufficientMemory" Jun 03 13:53:44.723160 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.723084 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 13:53:44.723160 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.723102 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeHasSufficientPID" Jun 03 13:53:44.740864 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.740848 2565 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-89.ec2.internal\" not found" node="ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.744257 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.744233 2565 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-89.ec2.internal\" not found" node="ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.782647 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.782631 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-89.ec2.internal\" not found" Jun 03 13:53:44.883224 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.883193 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-89.ec2.internal\" not found" Jun 03 13:53:44.897583 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.897563 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fefce7aa3fbfbe1683a83e63c8f3a9b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal\" (UID: \"fefce7aa3fbfbe1683a83e63c8f3a9b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.897644 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.897589 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fefce7aa3fbfbe1683a83e63c8f3a9b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal\" (UID: \"fefce7aa3fbfbe1683a83e63c8f3a9b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.897644 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.897606 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/92c7e5b782277de57331bd32a595193d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-89.ec2.internal\" (UID: \"92c7e5b782277de57331bd32a595193d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.984015 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:44.983990 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-89.ec2.internal\" not found" Jun 03 13:53:44.998432 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.998415 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/92c7e5b782277de57331bd32a595193d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-89.ec2.internal\" (UID: \"92c7e5b782277de57331bd32a595193d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.998482 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.998438 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fefce7aa3fbfbe1683a83e63c8f3a9b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal\" (UID: \"fefce7aa3fbfbe1683a83e63c8f3a9b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.998482 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.998455 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fefce7aa3fbfbe1683a83e63c8f3a9b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal\" (UID: \"fefce7aa3fbfbe1683a83e63c8f3a9b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.998558 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.998493 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fefce7aa3fbfbe1683a83e63c8f3a9b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal\" (UID: \"fefce7aa3fbfbe1683a83e63c8f3a9b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.998558 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.998517 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/92c7e5b782277de57331bd32a595193d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-89.ec2.internal\" (UID: \"92c7e5b782277de57331bd32a595193d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-89.ec2.internal" Jun 03 13:53:44.998558 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:44.998516 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fefce7aa3fbfbe1683a83e63c8f3a9b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal\" (UID: \"fefce7aa3fbfbe1683a83e63c8f3a9b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal" Jun 03 13:53:45.042465 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.042445 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal" Jun 03 13:53:45.046968 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.046951 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-89.ec2.internal" Jun 03 13:53:45.084353 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:45.084317 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-89.ec2.internal\" not found" Jun 03 13:53:45.184923 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:45.184888 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-89.ec2.internal\" not found" Jun 03 13:53:45.285569 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:45.285502 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-89.ec2.internal\" not found" Jun 03 13:53:45.386264 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:45.386217 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-89.ec2.internal\" not found" Jun 03 13:53:45.404766 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.404743 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 03 13:53:45.404899 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.404884 2565 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 13:53:45.487364 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:45.487330 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-89.ec2.internal\" not found" Jun 03 13:53:45.494464 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.494449 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 03 13:53:45.508400 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.508381 2565 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 13:53:45.511911 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.511894 2565 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 13:53:45.532913 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.532887 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-02 13:48:44 +0000 UTC" deadline="2027-12-24 17:31:16.667745498 +0000 UTC" Jun 03 13:53:45.532913 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.532911 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13659h37m31.134837238s" Jun 03 13:53:45.534427 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.534411 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pdtl9" Jun 03 13:53:45.542775 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.542730 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pdtl9" Jun 03 13:53:45.587544 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:45.587523 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-89.ec2.internal\" not found" Jun 03 13:53:45.590101 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.590083 2565 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 13:53:45.609704 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:45.609671 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfefce7aa3fbfbe1683a83e63c8f3a9b9.slice/crio-7e08db379820ee4ec17fb1fe9a53b7c4f66625d8bd30542c3f4ec60f8086b75c WatchSource:0}: Error finding container 7e08db379820ee4ec17fb1fe9a53b7c4f66625d8bd30542c3f4ec60f8086b75c: Status 404 returned error can't find the container with id 7e08db379820ee4ec17fb1fe9a53b7c4f66625d8bd30542c3f4ec60f8086b75c Jun 03 13:53:45.611309 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:45.611289 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c7e5b782277de57331bd32a595193d.slice/crio-d80587e4048d665d894bbf059504ac029f4794cf620b05bf1d25573d34c14d5f WatchSource:0}: Error finding container d80587e4048d665d894bbf059504ac029f4794cf620b05bf1d25573d34c14d5f: Status 404 returned error can't find the container with id d80587e4048d665d894bbf059504ac029f4794cf620b05bf1d25573d34c14d5f Jun 03 13:53:45.615999 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.615983 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 13:53:45.621200 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.621154 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-89.ec2.internal" event={"ID":"92c7e5b782277de57331bd32a595193d","Type":"ContainerStarted","Data":"d80587e4048d665d894bbf059504ac029f4794cf620b05bf1d25573d34c14d5f"} Jun 03 13:53:45.622029 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.622010 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal" event={"ID":"fefce7aa3fbfbe1683a83e63c8f3a9b9","Type":"ContainerStarted","Data":"7e08db379820ee4ec17fb1fe9a53b7c4f66625d8bd30542c3f4ec60f8086b75c"} Jun 03 13:53:45.680083 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.680058 2565 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 13:53:45.694646 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.694627 2565 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal" Jun 03 13:53:45.710791 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.710774 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 13:53:45.711863 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.711851 2565 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-89.ec2.internal" Jun 03 13:53:45.728409 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:45.728391 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 13:53:46.472152 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.472119 2565 apiserver.go:52] "Watching apiserver" Jun 03 13:53:46.481979 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.481954 2565 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 03 13:53:46.484632 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.484595 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-zp6hm","openshift-multus/network-metrics-daemon-5m5l8","openshift-network-diagnostics/network-check-target-qsp8b","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx","openshift-dns/node-resolver-gbtxm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal","openshift-multus/multus-additional-cni-plugins-f5chv","openshift-network-operator/iptables-alerter-nbbzd","openshift-ovn-kubernetes/ovnkube-node-gzz7s","kube-system/konnectivity-agent-np4q9","kube-system/kube-apiserver-proxy-ip-10-0-134-89.ec2.internal","openshift-cluster-node-tuning-operator/tuned-r4ntw","openshift-image-registry/node-ca-6ns2r"] Jun 03 13:53:46.489424 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.489198 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-np4q9" Jun 03 13:53:46.491251 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.491177 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:46.491372 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:46.491261 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:53:46.491372 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.491283 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:46.491492 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:46.491410 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:53:46.493085 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.492796 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 03 13:53:46.493680 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.493395 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 03 13:53:46.493680 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.493630 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tqscp\"" Jun 03 13:53:46.495420 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.495401 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.498044 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.497666 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gbtxm" Jun 03 13:53:46.500491 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.499776 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nbbzd" Jun 03 13:53:46.500992 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.500908 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-prpmb\"" Jun 03 13:53:46.501167 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.501143 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 03 13:53:46.501295 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.501276 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qgvgc\"" Jun 03 13:53:46.501295 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.501144 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 03 13:53:46.501295 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.501291 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 03 13:53:46.501521 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.501392 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 03 13:53:46.501521 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.501146 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 03 13:53:46.502364 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.502331 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.502451 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.502415 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z86xz\"" Jun 03 13:53:46.504696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.504538 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 03 13:53:46.505165 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505141 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-registration-dir\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.505251 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505180 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-etc-selinux\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.505251 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505206 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-sys-fs\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.505251 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505228 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c5d4af8-6b15-491f-9c16-884f366947e3-hosts-file\") pod \"node-resolver-gbtxm\" (UID: \"8c5d4af8-6b15-491f-9c16-884f366947e3\") " pod="openshift-dns/node-resolver-gbtxm" Jun 03 13:53:46.505391 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505253 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c5d4af8-6b15-491f-9c16-884f366947e3-tmp-dir\") pod \"node-resolver-gbtxm\" (UID: \"8c5d4af8-6b15-491f-9c16-884f366947e3\") " pod="openshift-dns/node-resolver-gbtxm" Jun 03 13:53:46.505391 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505278 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/77b40cee-f20f-4399-b4ff-42fc5f16335e-agent-certs\") pod \"konnectivity-agent-np4q9\" (UID: \"77b40cee-f20f-4399-b4ff-42fc5f16335e\") " pod="kube-system/konnectivity-agent-np4q9" Jun 03 13:53:46.505391 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505302 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:46.505391 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505332 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksbrc\" (UniqueName: \"kubernetes.io/projected/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-kube-api-access-ksbrc\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:46.505391 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505379 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.505559 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505402 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-device-dir\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.505559 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505425 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jcxg\" (UniqueName: \"kubernetes.io/projected/8c5d4af8-6b15-491f-9c16-884f366947e3-kube-api-access-7jcxg\") pod \"node-resolver-gbtxm\" (UID: \"8c5d4af8-6b15-491f-9c16-884f366947e3\") " pod="openshift-dns/node-resolver-gbtxm" Jun 03 13:53:46.505559 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505448 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b8242da-a838-4502-88d1-ecc03b3386ce-iptables-alerter-script\") pod \"iptables-alerter-nbbzd\" (UID: \"4b8242da-a838-4502-88d1-ecc03b3386ce\") " pod="openshift-network-operator/iptables-alerter-nbbzd" Jun 03 13:53:46.505559 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505474 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5vvg\" (UniqueName: \"kubernetes.io/projected/cc27edab-d11b-4e37-a749-9a7ca106e24e-kube-api-access-l5vvg\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.505559 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505504 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b8242da-a838-4502-88d1-ecc03b3386ce-host-slash\") pod \"iptables-alerter-nbbzd\" (UID: \"4b8242da-a838-4502-88d1-ecc03b3386ce\") " pod="openshift-network-operator/iptables-alerter-nbbzd" Jun 03 13:53:46.505559 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505529 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgtps\" (UniqueName: \"kubernetes.io/projected/4b8242da-a838-4502-88d1-ecc03b3386ce-kube-api-access-fgtps\") pod \"iptables-alerter-nbbzd\" (UID: \"4b8242da-a838-4502-88d1-ecc03b3386ce\") " pod="openshift-network-operator/iptables-alerter-nbbzd" Jun 03 13:53:46.505743 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505561 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/77b40cee-f20f-4399-b4ff-42fc5f16335e-konnectivity-ca\") pod \"konnectivity-agent-np4q9\" (UID: \"77b40cee-f20f-4399-b4ff-42fc5f16335e\") " pod="kube-system/konnectivity-agent-np4q9" Jun 03 13:53:46.505743 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505586 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h84w6\" (UniqueName: \"kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6\") pod \"network-check-target-qsp8b\" (UID: \"8eb13e0d-051c-4a0e-a19d-6252080682a9\") " pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:46.505743 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505615 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-socket-dir\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.505862 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.505755 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.510008 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.509983 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6ns2r" Jun 03 13:53:46.512773 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.512755 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.513637 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.513614 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.514280 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.514262 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 03 13:53:46.514694 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.514671 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 03 13:53:46.514932 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.514913 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 03 13:53:46.515172 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.515154 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 03 13:53:46.515248 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.515189 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 03 13:53:46.515371 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.515352 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p2wfx\"" Jun 03 13:53:46.515562 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.515544 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 03 13:53:46.517402 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.515952 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 03 13:53:46.517402 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.516372 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 03 13:53:46.517402 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.516414 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 03 13:53:46.517402 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.516433 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 03 13:53:46.517402 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.516620 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 03 13:53:46.517402 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.516747 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 03 13:53:46.517402 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.516802 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 03 13:53:46.517402 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.516748 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 03 13:53:46.517402 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.517001 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 03 13:53:46.517402 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.517129 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tfptg\"" Jun 03 13:53:46.517402 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.517181 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wxjcz\"" Jun 03 13:53:46.517402 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.517264 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 03 13:53:46.518017 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.517524 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 03 13:53:46.518305 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.518164 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gcsrd\"" Jun 03 13:53:46.518672 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.518512 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 03 13:53:46.522107 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.522087 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gkqg5\"" Jun 03 13:53:46.522250 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.522231 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 03 13:53:46.544144 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.544115 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 13:48:45 +0000 UTC" deadline="2027-11-04 18:42:30.420291004 +0000 UTC" Jun 03 13:53:46.544232 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.544145 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12460h48m43.876150128s" Jun 03 13:53:46.597710 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.597685 2565 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 03 13:53:46.606159 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606132 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46b08401-ddce-482b-9907-33302b219ef2-tmp\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.606287 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606181 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b8242da-a838-4502-88d1-ecc03b3386ce-iptables-alerter-script\") pod \"iptables-alerter-nbbzd\" (UID: \"4b8242da-a838-4502-88d1-ecc03b3386ce\") " pod="openshift-network-operator/iptables-alerter-nbbzd" Jun 03 13:53:46.606287 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606238 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7vth\" (UniqueName: \"kubernetes.io/projected/1d997ce2-6758-4937-8e63-dad6bf7b689a-kube-api-access-k7vth\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.606417 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606282 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-var-lib-openvswitch\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.606417 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606309 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-lib-modules\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.606417 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606333 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dedc278e-adec-4d58-bafb-047e9a36c84c-serviceca\") pod \"node-ca-6ns2r\" (UID: \"dedc278e-adec-4d58-bafb-047e9a36c84c\") " pod="openshift-image-registry/node-ca-6ns2r" Jun 03 13:53:46.606417 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606399 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l5vvg\" (UniqueName: \"kubernetes.io/projected/cc27edab-d11b-4e37-a749-9a7ca106e24e-kube-api-access-l5vvg\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.606610 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606442 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d997ce2-6758-4937-8e63-dad6bf7b689a-system-cni-dir\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.606610 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606552 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-run-netns\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.606610 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606580 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-cni-netd\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.606610 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606606 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.606799 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606665 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d49f3147-aee7-4a57-88f6-ad00854e9eb3-env-overrides\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.606799 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606710 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-systemd\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.606799 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606741 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klwk9\" (UniqueName: \"kubernetes.io/projected/dedc278e-adec-4d58-bafb-047e9a36c84c-kube-api-access-klwk9\") pod \"node-ca-6ns2r\" (UID: \"dedc278e-adec-4d58-bafb-047e9a36c84c\") " pod="openshift-image-registry/node-ca-6ns2r" Jun 03 13:53:46.606799 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606766 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b8242da-a838-4502-88d1-ecc03b3386ce-iptables-alerter-script\") pod \"iptables-alerter-nbbzd\" (UID: \"4b8242da-a838-4502-88d1-ecc03b3386ce\") " pod="openshift-network-operator/iptables-alerter-nbbzd" Jun 03 13:53:46.606799 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606773 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h84w6\" (UniqueName: \"kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6\") pod \"network-check-target-qsp8b\" (UID: \"8eb13e0d-051c-4a0e-a19d-6252080682a9\") " pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:46.607030 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606801 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d997ce2-6758-4937-8e63-dad6bf7b689a-cni-binary-copy\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.607030 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606836 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4df1db34-2741-48c1-844e-231fbafcd068-cni-binary-copy\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.607030 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606863 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dncn5\" (UniqueName: \"kubernetes.io/projected/4df1db34-2741-48c1-844e-231fbafcd068-kube-api-access-dncn5\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.607030 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606890 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-sysctl-d\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.607030 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606915 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/46b08401-ddce-482b-9907-33302b219ef2-etc-tuned\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.607030 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606937 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dedc278e-adec-4d58-bafb-047e9a36c84c-host\") pod \"node-ca-6ns2r\" (UID: \"dedc278e-adec-4d58-bafb-047e9a36c84c\") " pod="openshift-image-registry/node-ca-6ns2r" Jun 03 13:53:46.607030 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606964 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-socket-dir\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.607030 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.606990 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-sys-fs\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.607030 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607013 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-system-cni-dir\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607038 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-run-k8s-cni-cncf-io\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607062 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607087 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607100 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-sys-fs\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607113 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7jcxg\" (UniqueName: \"kubernetes.io/projected/8c5d4af8-6b15-491f-9c16-884f366947e3-kube-api-access-7jcxg\") pod \"node-resolver-gbtxm\" (UID: \"8c5d4af8-6b15-491f-9c16-884f366947e3\") " pod="openshift-dns/node-resolver-gbtxm" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607108 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-socket-dir\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607150 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-run-systemd\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607179 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:46.607212 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607249 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b8242da-a838-4502-88d1-ecc03b3386ce-host-slash\") pod \"iptables-alerter-nbbzd\" (UID: \"4b8242da-a838-4502-88d1-ecc03b3386ce\") " pod="openshift-network-operator/iptables-alerter-nbbzd" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607293 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b8242da-a838-4502-88d1-ecc03b3386ce-host-slash\") pod \"iptables-alerter-nbbzd\" (UID: \"4b8242da-a838-4502-88d1-ecc03b3386ce\") " pod="openshift-network-operator/iptables-alerter-nbbzd" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:46.607311 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs podName:277e2d8b-f5ab-4f08-bb7e-b6981b3932fc nodeName:}" failed. No retries permitted until 2026-06-03 13:53:47.107255567 +0000 UTC m=+3.051899526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs") pod "network-metrics-daemon-5m5l8" (UID: "277e2d8b-f5ab-4f08-bb7e-b6981b3932fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607366 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4df1db34-2741-48c1-844e-231fbafcd068-multus-daemon-config\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607391 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-run\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607412 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-sys\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.607456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607432 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-os-release\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607464 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1d997ce2-6758-4937-8e63-dad6bf7b689a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607494 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c5d4af8-6b15-491f-9c16-884f366947e3-tmp-dir\") pod \"node-resolver-gbtxm\" (UID: \"8c5d4af8-6b15-491f-9c16-884f366947e3\") " pod="openshift-dns/node-resolver-gbtxm" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607536 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-etc-openvswitch\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607587 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d49f3147-aee7-4a57-88f6-ad00854e9eb3-ovn-node-metrics-cert\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607614 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-run-netns\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607654 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-hostroot\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607713 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vc5g\" (UniqueName: \"kubernetes.io/projected/46b08401-ddce-482b-9907-33302b219ef2-kube-api-access-9vc5g\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607746 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ksbrc\" (UniqueName: \"kubernetes.io/projected/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-kube-api-access-ksbrc\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607802 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c5d4af8-6b15-491f-9c16-884f366947e3-tmp-dir\") pod \"node-resolver-gbtxm\" (UID: \"8c5d4af8-6b15-491f-9c16-884f366947e3\") " pod="openshift-dns/node-resolver-gbtxm" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607807 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fgtps\" (UniqueName: \"kubernetes.io/projected/4b8242da-a838-4502-88d1-ecc03b3386ce-kube-api-access-fgtps\") pod \"iptables-alerter-nbbzd\" (UID: \"4b8242da-a838-4502-88d1-ecc03b3386ce\") " pod="openshift-network-operator/iptables-alerter-nbbzd" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607865 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-run-openvswitch\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607882 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d49f3147-aee7-4a57-88f6-ad00854e9eb3-ovnkube-config\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607921 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-cnibin\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607949 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-sysconfig\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607964 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-multus-socket-dir-parent\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.607989 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/77b40cee-f20f-4399-b4ff-42fc5f16335e-konnectivity-ca\") pod \"konnectivity-agent-np4q9\" (UID: \"77b40cee-f20f-4399-b4ff-42fc5f16335e\") " pod="kube-system/konnectivity-agent-np4q9" Jun 03 13:53:46.608199 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608023 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d997ce2-6758-4937-8e63-dad6bf7b689a-cnibin\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608063 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-systemd-units\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608095 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-log-socket\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608119 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-run-multus-certs\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608142 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-modprobe-d\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608165 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-sysctl-conf\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608218 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-cni-bin\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608248 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/77b40cee-f20f-4399-b4ff-42fc5f16335e-agent-certs\") pod \"konnectivity-agent-np4q9\" (UID: \"77b40cee-f20f-4399-b4ff-42fc5f16335e\") " pod="kube-system/konnectivity-agent-np4q9" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608275 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-device-dir\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608300 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-kubelet\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608325 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-run-ovn\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608378 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqsw2\" (UniqueName: \"kubernetes.io/projected/d49f3147-aee7-4a57-88f6-ad00854e9eb3-kube-api-access-kqsw2\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608394 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-device-dir\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608402 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-var-lib-kubelet\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608439 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-var-lib-kubelet\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608468 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d997ce2-6758-4937-8e63-dad6bf7b689a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608470 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/77b40cee-f20f-4399-b4ff-42fc5f16335e-konnectivity-ca\") pod \"konnectivity-agent-np4q9\" (UID: \"77b40cee-f20f-4399-b4ff-42fc5f16335e\") " pod="kube-system/konnectivity-agent-np4q9" Jun 03 13:53:46.608981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608491 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-var-lib-cni-bin\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608536 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-var-lib-cni-multus\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608559 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-multus-conf-dir\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608577 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608581 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-etc-kubernetes\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608613 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-kubernetes\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608634 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-host\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608657 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d997ce2-6758-4937-8e63-dad6bf7b689a-os-release\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608691 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-node-log\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608715 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-run-ovn-kubernetes\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608739 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d49f3147-aee7-4a57-88f6-ad00854e9eb3-ovnkube-script-lib\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608801 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-registration-dir\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608827 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-etc-selinux\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608853 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c5d4af8-6b15-491f-9c16-884f366947e3-hosts-file\") pod \"node-resolver-gbtxm\" (UID: \"8c5d4af8-6b15-491f-9c16-884f366947e3\") " pod="openshift-dns/node-resolver-gbtxm" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608879 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1d997ce2-6758-4937-8e63-dad6bf7b689a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608878 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-registration-dir\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608928 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8c5d4af8-6b15-491f-9c16-884f366947e3-hosts-file\") pod \"node-resolver-gbtxm\" (UID: \"8c5d4af8-6b15-491f-9c16-884f366947e3\") " pod="openshift-dns/node-resolver-gbtxm" Jun 03 13:53:46.609822 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608964 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-slash\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.610607 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608984 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cc27edab-d11b-4e37-a749-9a7ca106e24e-etc-selinux\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.610607 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.608995 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-multus-cni-dir\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.612472 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.612454 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/77b40cee-f20f-4399-b4ff-42fc5f16335e-agent-certs\") pod \"konnectivity-agent-np4q9\" (UID: \"77b40cee-f20f-4399-b4ff-42fc5f16335e\") " pod="kube-system/konnectivity-agent-np4q9" Jun 03 13:53:46.617221 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:46.617199 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:53:46.617221 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:46.617223 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:53:46.617381 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:46.617235 2565 projected.go:194] Error preparing data for projected volume kube-api-access-h84w6 for pod openshift-network-diagnostics/network-check-target-qsp8b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:53:46.617381 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:46.617289 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6 podName:8eb13e0d-051c-4a0e-a19d-6252080682a9 nodeName:}" failed. No retries permitted until 2026-06-03 13:53:47.117272592 +0000 UTC m=+3.061916540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h84w6" (UniqueName: "kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6") pod "network-check-target-qsp8b" (UID: "8eb13e0d-051c-4a0e-a19d-6252080682a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:53:46.621202 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.620130 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksbrc\" (UniqueName: \"kubernetes.io/projected/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-kube-api-access-ksbrc\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:46.621202 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.620594 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jcxg\" (UniqueName: \"kubernetes.io/projected/8c5d4af8-6b15-491f-9c16-884f366947e3-kube-api-access-7jcxg\") pod \"node-resolver-gbtxm\" (UID: \"8c5d4af8-6b15-491f-9c16-884f366947e3\") " pod="openshift-dns/node-resolver-gbtxm" Jun 03 13:53:46.621202 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.620879 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5vvg\" (UniqueName: \"kubernetes.io/projected/cc27edab-d11b-4e37-a749-9a7ca106e24e-kube-api-access-l5vvg\") pod \"aws-ebs-csi-driver-node-2l7qx\" (UID: \"cc27edab-d11b-4e37-a749-9a7ca106e24e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.622909 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.622888 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgtps\" (UniqueName: \"kubernetes.io/projected/4b8242da-a838-4502-88d1-ecc03b3386ce-kube-api-access-fgtps\") pod \"iptables-alerter-nbbzd\" (UID: \"4b8242da-a838-4502-88d1-ecc03b3386ce\") " pod="openshift-network-operator/iptables-alerter-nbbzd" Jun 03 13:53:46.666960 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.666931 2565 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 13:53:46.710112 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710079 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k7vth\" (UniqueName: \"kubernetes.io/projected/1d997ce2-6758-4937-8e63-dad6bf7b689a-kube-api-access-k7vth\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.710262 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710120 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-var-lib-openvswitch\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.710262 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710174 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-var-lib-openvswitch\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.710399 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710278 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-lib-modules\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.710399 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710304 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dedc278e-adec-4d58-bafb-047e9a36c84c-serviceca\") pod \"node-ca-6ns2r\" (UID: \"dedc278e-adec-4d58-bafb-047e9a36c84c\") " pod="openshift-image-registry/node-ca-6ns2r" Jun 03 13:53:46.710399 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710328 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d997ce2-6758-4937-8e63-dad6bf7b689a-system-cni-dir\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.710514 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710400 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-run-netns\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.710514 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710455 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-cni-netd\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.710583 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710507 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.710615 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710574 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d49f3147-aee7-4a57-88f6-ad00854e9eb3-env-overrides\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.710666 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710617 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-systemd\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.710717 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710668 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-klwk9\" (UniqueName: \"kubernetes.io/projected/dedc278e-adec-4d58-bafb-047e9a36c84c-kube-api-access-klwk9\") pod \"node-ca-6ns2r\" (UID: \"dedc278e-adec-4d58-bafb-047e9a36c84c\") " pod="openshift-image-registry/node-ca-6ns2r" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710764 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d997ce2-6758-4937-8e63-dad6bf7b689a-cni-binary-copy\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710808 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dedc278e-adec-4d58-bafb-047e9a36c84c-serviceca\") pod \"node-ca-6ns2r\" (UID: \"dedc278e-adec-4d58-bafb-047e9a36c84c\") " pod="openshift-image-registry/node-ca-6ns2r" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710812 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4df1db34-2741-48c1-844e-231fbafcd068-cni-binary-copy\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710827 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-run-netns\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710827 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710860 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dncn5\" (UniqueName: \"kubernetes.io/projected/4df1db34-2741-48c1-844e-231fbafcd068-kube-api-access-dncn5\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710863 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-cni-netd\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710884 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1d997ce2-6758-4937-8e63-dad6bf7b689a-system-cni-dir\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710901 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-sysctl-d\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710952 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/46b08401-ddce-482b-9907-33302b219ef2-etc-tuned\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710976 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dedc278e-adec-4d58-bafb-047e9a36c84c-host\") pod \"node-ca-6ns2r\" (UID: \"dedc278e-adec-4d58-bafb-047e9a36c84c\") " pod="openshift-image-registry/node-ca-6ns2r" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.710968 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-systemd\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711024 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-system-cni-dir\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711221 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-run-k8s-cni-cncf-io\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711266 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-run-systemd\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711295 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4df1db34-2741-48c1-844e-231fbafcd068-multus-daemon-config\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711319 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-run\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.711523 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711361 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-sys\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711386 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-os-release\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711384 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-run-k8s-cni-cncf-io\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711411 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1d997ce2-6758-4937-8e63-dad6bf7b689a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711437 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dedc278e-adec-4d58-bafb-047e9a36c84c-host\") pod \"node-ca-6ns2r\" (UID: \"dedc278e-adec-4d58-bafb-047e9a36c84c\") " pod="openshift-image-registry/node-ca-6ns2r" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711444 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-etc-openvswitch\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711470 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-sys\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711556 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1d997ce2-6758-4937-8e63-dad6bf7b689a-cni-binary-copy\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711559 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4df1db34-2741-48c1-844e-231fbafcd068-cni-binary-copy\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711592 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-etc-openvswitch\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711633 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-system-cni-dir\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711649 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-sysctl-d\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711655 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-lib-modules\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711677 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-run\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711690 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-run-systemd\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711716 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-os-release\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711728 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d49f3147-aee7-4a57-88f6-ad00854e9eb3-ovn-node-metrics-cert\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711757 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-run-netns\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.712431 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711756 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d49f3147-aee7-4a57-88f6-ad00854e9eb3-env-overrides\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711783 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-hostroot\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711789 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-run-netns\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711807 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9vc5g\" (UniqueName: \"kubernetes.io/projected/46b08401-ddce-482b-9907-33302b219ef2-kube-api-access-9vc5g\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711835 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-run-openvswitch\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711844 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-hostroot\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711871 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d49f3147-aee7-4a57-88f6-ad00854e9eb3-ovnkube-config\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711878 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-run-openvswitch\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.711960 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4df1db34-2741-48c1-844e-231fbafcd068-multus-daemon-config\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712044 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-cnibin\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712074 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-sysconfig\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712100 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-multus-socket-dir-parent\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712099 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1d997ce2-6758-4937-8e63-dad6bf7b689a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712130 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d997ce2-6758-4937-8e63-dad6bf7b689a-cnibin\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712147 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-cnibin\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712176 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1d997ce2-6758-4937-8e63-dad6bf7b689a-cnibin\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712182 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-multus-socket-dir-parent\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712162 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-sysconfig\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.713234 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712215 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-systemd-units\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712240 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-log-socket\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712263 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-systemd-units\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712265 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-run-multus-certs\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712292 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-log-socket\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712302 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-run-multus-certs\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712306 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-modprobe-d\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712334 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-sysctl-conf\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712373 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-cni-bin\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712328 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d49f3147-aee7-4a57-88f6-ad00854e9eb3-ovnkube-config\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712402 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-kubelet\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712402 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-modprobe-d\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712439 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-kubelet\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712449 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-cni-bin\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712467 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-run-ovn\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712472 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-sysctl-conf\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712501 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kqsw2\" (UniqueName: \"kubernetes.io/projected/d49f3147-aee7-4a57-88f6-ad00854e9eb3-kube-api-access-kqsw2\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712521 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-run-ovn\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.714094 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712527 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-var-lib-kubelet\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712553 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-var-lib-kubelet\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712564 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-var-lib-kubelet\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712581 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d997ce2-6758-4937-8e63-dad6bf7b689a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712605 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-var-lib-cni-bin\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712629 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-var-lib-cni-multus\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712652 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-var-lib-kubelet\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712653 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-multus-conf-dir\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712691 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-etc-kubernetes\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712689 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-multus-conf-dir\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712720 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-kubernetes\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712726 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1d997ce2-6758-4937-8e63-dad6bf7b689a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712726 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-var-lib-cni-bin\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712740 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-host-var-lib-cni-multus\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712744 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-host\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712759 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-etc-kubernetes\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712787 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d997ce2-6758-4937-8e63-dad6bf7b689a-os-release\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712792 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-host\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.714945 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712790 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46b08401-ddce-482b-9907-33302b219ef2-etc-kubernetes\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712815 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-node-log\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712837 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1d997ce2-6758-4937-8e63-dad6bf7b689a-os-release\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712840 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-run-ovn-kubernetes\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712873 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d49f3147-aee7-4a57-88f6-ad00854e9eb3-ovnkube-script-lib\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712877 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-node-log\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712906 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1d997ce2-6758-4937-8e63-dad6bf7b689a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712908 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-run-ovn-kubernetes\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712936 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-slash\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712963 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-multus-cni-dir\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712991 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d49f3147-aee7-4a57-88f6-ad00854e9eb3-host-slash\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.712993 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46b08401-ddce-482b-9907-33302b219ef2-tmp\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.713099 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4df1db34-2741-48c1-844e-231fbafcd068-multus-cni-dir\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.713402 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1d997ce2-6758-4937-8e63-dad6bf7b689a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.714439 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/46b08401-ddce-482b-9907-33302b219ef2-etc-tuned\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.715289 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d49f3147-aee7-4a57-88f6-ad00854e9eb3-ovn-node-metrics-cert\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.715318 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46b08401-ddce-482b-9907-33302b219ef2-tmp\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.715696 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.715571 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d49f3147-aee7-4a57-88f6-ad00854e9eb3-ovnkube-script-lib\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.719143 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.719118 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7vth\" (UniqueName: \"kubernetes.io/projected/1d997ce2-6758-4937-8e63-dad6bf7b689a-kube-api-access-k7vth\") pod \"multus-additional-cni-plugins-f5chv\" (UID: \"1d997ce2-6758-4937-8e63-dad6bf7b689a\") " pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.719843 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.719823 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dncn5\" (UniqueName: \"kubernetes.io/projected/4df1db34-2741-48c1-844e-231fbafcd068-kube-api-access-dncn5\") pod \"multus-zp6hm\" (UID: \"4df1db34-2741-48c1-844e-231fbafcd068\") " pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.721436 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.721412 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klwk9\" (UniqueName: \"kubernetes.io/projected/dedc278e-adec-4d58-bafb-047e9a36c84c-kube-api-access-klwk9\") pod \"node-ca-6ns2r\" (UID: \"dedc278e-adec-4d58-bafb-047e9a36c84c\") " pod="openshift-image-registry/node-ca-6ns2r" Jun 03 13:53:46.722268 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.722219 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vc5g\" (UniqueName: \"kubernetes.io/projected/46b08401-ddce-482b-9907-33302b219ef2-kube-api-access-9vc5g\") pod \"tuned-r4ntw\" (UID: \"46b08401-ddce-482b-9907-33302b219ef2\") " pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:46.722420 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.722401 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqsw2\" (UniqueName: \"kubernetes.io/projected/d49f3147-aee7-4a57-88f6-ad00854e9eb3-kube-api-access-kqsw2\") pod \"ovnkube-node-gzz7s\" (UID: \"d49f3147-aee7-4a57-88f6-ad00854e9eb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.803378 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.803334 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-np4q9" Jun 03 13:53:46.820298 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.820267 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" Jun 03 13:53:46.829869 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.829849 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gbtxm" Jun 03 13:53:46.835396 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.835380 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nbbzd" Jun 03 13:53:46.842917 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.842889 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zp6hm" Jun 03 13:53:46.850491 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.850474 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:53:46.857070 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.857048 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6ns2r" Jun 03 13:53:46.865772 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.865749 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f5chv" Jun 03 13:53:46.872296 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:46.872279 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" Jun 03 13:53:47.116320 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.116231 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:47.116490 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:47.116386 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:53:47.116490 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:47.116460 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs podName:277e2d8b-f5ab-4f08-bb7e-b6981b3932fc nodeName:}" failed. No retries permitted until 2026-06-03 13:53:48.116440327 +0000 UTC m=+4.061084263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs") pod "network-metrics-daemon-5m5l8" (UID: "277e2d8b-f5ab-4f08-bb7e-b6981b3932fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:53:47.217553 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.217518 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h84w6\" (UniqueName: \"kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6\") pod \"network-check-target-qsp8b\" (UID: \"8eb13e0d-051c-4a0e-a19d-6252080682a9\") " pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:47.217717 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:47.217656 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:53:47.217717 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:47.217678 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:53:47.217717 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:47.217692 2565 projected.go:194] Error preparing data for projected volume kube-api-access-h84w6 for pod openshift-network-diagnostics/network-check-target-qsp8b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:53:47.217853 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:47.217782 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6 podName:8eb13e0d-051c-4a0e-a19d-6252080682a9 nodeName:}" failed. No retries permitted until 2026-06-03 13:53:48.217761855 +0000 UTC m=+4.162405791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h84w6" (UniqueName: "kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6") pod "network-check-target-qsp8b" (UID: "8eb13e0d-051c-4a0e-a19d-6252080682a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:53:47.544451 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.544409 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 13:48:45 +0000 UTC" deadline="2027-12-11 17:18:34.556106382 +0000 UTC" Jun 03 13:53:47.544451 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.544445 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13347h24m47.011663822s" Jun 03 13:53:47.575955 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:47.575930 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc27edab_d11b_4e37_a749_9a7ca106e24e.slice/crio-d6f53522245f322166a9cfd03a9499db726c802b206543b717d7f6012f7a2fe7 WatchSource:0}: Error finding container d6f53522245f322166a9cfd03a9499db726c802b206543b717d7f6012f7a2fe7: Status 404 returned error can't find the container with id d6f53522245f322166a9cfd03a9499db726c802b206543b717d7f6012f7a2fe7 Jun 03 13:53:47.576930 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:47.576907 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c5d4af8_6b15_491f_9c16_884f366947e3.slice/crio-a2ccad90c24cd1c49e753b49825c1824904741bd927b21adfe95d4d5c493a989 WatchSource:0}: Error finding container a2ccad90c24cd1c49e753b49825c1824904741bd927b21adfe95d4d5c493a989: Status 404 returned error can't find the container with id a2ccad90c24cd1c49e753b49825c1824904741bd927b21adfe95d4d5c493a989 Jun 03 13:53:47.577630 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:47.577536 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddedc278e_adec_4d58_bafb_047e9a36c84c.slice/crio-f97db524e681e8c14b548be11cf8fa2d1be4b2206d8563871ac03ea6c7192163 WatchSource:0}: Error finding container f97db524e681e8c14b548be11cf8fa2d1be4b2206d8563871ac03ea6c7192163: Status 404 returned error can't find the container with id f97db524e681e8c14b548be11cf8fa2d1be4b2206d8563871ac03ea6c7192163 Jun 03 13:53:47.579919 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:47.579471 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d997ce2_6758_4937_8e63_dad6bf7b689a.slice/crio-0fd6bc514a60a440dd249ae16fd7380e2c0a5dafc95117fb1ca760c65c6b2c61 WatchSource:0}: Error finding container 0fd6bc514a60a440dd249ae16fd7380e2c0a5dafc95117fb1ca760c65c6b2c61: Status 404 returned error can't find the container with id 0fd6bc514a60a440dd249ae16fd7380e2c0a5dafc95117fb1ca760c65c6b2c61 Jun 03 13:53:47.580853 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:47.580831 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df1db34_2741_48c1_844e_231fbafcd068.slice/crio-441aafff49e8f25f1b92b2bf36e0693e3bfe9e7ebf06ec2bb82bd8d118cdb325 WatchSource:0}: Error finding container 441aafff49e8f25f1b92b2bf36e0693e3bfe9e7ebf06ec2bb82bd8d118cdb325: Status 404 returned error can't find the container with id 441aafff49e8f25f1b92b2bf36e0693e3bfe9e7ebf06ec2bb82bd8d118cdb325 Jun 03 13:53:47.581883 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:47.581861 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd49f3147_aee7_4a57_88f6_ad00854e9eb3.slice/crio-ef326d5ccd817118b7e14f23141c6a30a91ddf8da2c8093f2261938ac429334c WatchSource:0}: Error finding container ef326d5ccd817118b7e14f23141c6a30a91ddf8da2c8093f2261938ac429334c: Status 404 returned error can't find the container with id ef326d5ccd817118b7e14f23141c6a30a91ddf8da2c8093f2261938ac429334c Jun 03 13:53:47.583059 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:47.582985 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b8242da_a838_4502_88d1_ecc03b3386ce.slice/crio-b18cc4e2bbb180afe71dd425df86172e1776360726a9f72dadb32e32323816d6 WatchSource:0}: Error finding container b18cc4e2bbb180afe71dd425df86172e1776360726a9f72dadb32e32323816d6: Status 404 returned error can't find the container with id b18cc4e2bbb180afe71dd425df86172e1776360726a9f72dadb32e32323816d6 Jun 03 13:53:47.585392 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:47.585333 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b40cee_f20f_4399_b4ff_42fc5f16335e.slice/crio-185da202c0358b6339b5039810a6a1f3b8f8b4a2a98c5a3e38fd2255a3a5fa4b WatchSource:0}: Error finding container 185da202c0358b6339b5039810a6a1f3b8f8b4a2a98c5a3e38fd2255a3a5fa4b: Status 404 returned error can't find the container with id 185da202c0358b6339b5039810a6a1f3b8f8b4a2a98c5a3e38fd2255a3a5fa4b Jun 03 13:53:47.585889 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:53:47.585848 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46b08401_ddce_482b_9907_33302b219ef2.slice/crio-a43769ca6346fa46f240b27242c505289a58a7a04462e332376b72b8374b974a WatchSource:0}: Error finding container a43769ca6346fa46f240b27242c505289a58a7a04462e332376b72b8374b974a: Status 404 returned error can't find the container with id a43769ca6346fa46f240b27242c505289a58a7a04462e332376b72b8374b974a Jun 03 13:53:47.618917 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.618790 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:47.619001 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:47.618983 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:53:47.628858 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.628825 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" event={"ID":"d49f3147-aee7-4a57-88f6-ad00854e9eb3","Type":"ContainerStarted","Data":"ef326d5ccd817118b7e14f23141c6a30a91ddf8da2c8093f2261938ac429334c"} Jun 03 13:53:47.629884 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.629862 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-np4q9" event={"ID":"77b40cee-f20f-4399-b4ff-42fc5f16335e","Type":"ContainerStarted","Data":"185da202c0358b6339b5039810a6a1f3b8f8b4a2a98c5a3e38fd2255a3a5fa4b"} Jun 03 13:53:47.630796 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.630773 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nbbzd" event={"ID":"4b8242da-a838-4502-88d1-ecc03b3386ce","Type":"ContainerStarted","Data":"b18cc4e2bbb180afe71dd425df86172e1776360726a9f72dadb32e32323816d6"} Jun 03 13:53:47.631644 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.631618 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zp6hm" event={"ID":"4df1db34-2741-48c1-844e-231fbafcd068","Type":"ContainerStarted","Data":"441aafff49e8f25f1b92b2bf36e0693e3bfe9e7ebf06ec2bb82bd8d118cdb325"} Jun 03 13:53:47.632537 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.632518 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6ns2r" event={"ID":"dedc278e-adec-4d58-bafb-047e9a36c84c","Type":"ContainerStarted","Data":"f97db524e681e8c14b548be11cf8fa2d1be4b2206d8563871ac03ea6c7192163"} Jun 03 13:53:47.633390 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.633367 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gbtxm" event={"ID":"8c5d4af8-6b15-491f-9c16-884f366947e3","Type":"ContainerStarted","Data":"a2ccad90c24cd1c49e753b49825c1824904741bd927b21adfe95d4d5c493a989"} Jun 03 13:53:47.637024 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.636998 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" event={"ID":"46b08401-ddce-482b-9907-33302b219ef2","Type":"ContainerStarted","Data":"a43769ca6346fa46f240b27242c505289a58a7a04462e332376b72b8374b974a"} Jun 03 13:53:47.637960 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.637940 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f5chv" event={"ID":"1d997ce2-6758-4937-8e63-dad6bf7b689a","Type":"ContainerStarted","Data":"0fd6bc514a60a440dd249ae16fd7380e2c0a5dafc95117fb1ca760c65c6b2c61"} Jun 03 13:53:47.638808 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:47.638791 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" event={"ID":"cc27edab-d11b-4e37-a749-9a7ca106e24e","Type":"ContainerStarted","Data":"d6f53522245f322166a9cfd03a9499db726c802b206543b717d7f6012f7a2fe7"} Jun 03 13:53:48.125247 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:48.125164 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:48.125431 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:48.125313 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:53:48.125431 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:48.125400 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs podName:277e2d8b-f5ab-4f08-bb7e-b6981b3932fc nodeName:}" failed. No retries permitted until 2026-06-03 13:53:50.125379713 +0000 UTC m=+6.070023669 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs") pod "network-metrics-daemon-5m5l8" (UID: "277e2d8b-f5ab-4f08-bb7e-b6981b3932fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:53:48.226355 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:48.226305 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h84w6\" (UniqueName: \"kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6\") pod \"network-check-target-qsp8b\" (UID: \"8eb13e0d-051c-4a0e-a19d-6252080682a9\") " pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:48.226603 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:48.226583 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:53:48.226688 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:48.226612 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:53:48.226688 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:48.226625 2565 projected.go:194] Error preparing data for projected volume kube-api-access-h84w6 for pod openshift-network-diagnostics/network-check-target-qsp8b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:53:48.226688 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:48.226688 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6 podName:8eb13e0d-051c-4a0e-a19d-6252080682a9 nodeName:}" failed. No retries permitted until 2026-06-03 13:53:50.226667141 +0000 UTC m=+6.171311075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h84w6" (UniqueName: "kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6") pod "network-check-target-qsp8b" (UID: "8eb13e0d-051c-4a0e-a19d-6252080682a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:53:48.622548 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:48.622032 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:48.622548 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:48.622174 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:53:48.647830 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:48.647177 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-89.ec2.internal" event={"ID":"92c7e5b782277de57331bd32a595193d","Type":"ContainerStarted","Data":"5460dfe1cdf7dd68a62670a4f217cbce6ef4408a07185fe15af2527c116b5c05"} Jun 03 13:53:48.650848 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:48.650763 2565 generic.go:358] "Generic (PLEG): container finished" podID="fefce7aa3fbfbe1683a83e63c8f3a9b9" containerID="dbdf61de678e0b01bf7043c229318002b7186bd1341ad134ded3ab032148aa6d" exitCode=0 Jun 03 13:53:48.650848 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:48.650811 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal" event={"ID":"fefce7aa3fbfbe1683a83e63c8f3a9b9","Type":"ContainerDied","Data":"dbdf61de678e0b01bf7043c229318002b7186bd1341ad134ded3ab032148aa6d"} Jun 03 13:53:48.678800 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:48.677901 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-89.ec2.internal" podStartSLOduration=3.677885388 podStartE2EDuration="3.677885388s" podCreationTimestamp="2026-06-03 13:53:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 13:53:48.66429803 +0000 UTC m=+4.608941985" watchObservedRunningTime="2026-06-03 13:53:48.677885388 +0000 UTC m=+4.622529345" Jun 03 13:53:49.620055 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:49.619583 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:49.620055 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:49.619706 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:53:49.664934 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:49.664874 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal" event={"ID":"fefce7aa3fbfbe1683a83e63c8f3a9b9","Type":"ContainerStarted","Data":"a8d80038d2c750f9695daef13753c039785fd8b9def5c3677621370ac4b2d03d"} Jun 03 13:53:49.686011 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:49.685237 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-89.ec2.internal" podStartSLOduration=4.685219135 podStartE2EDuration="4.685219135s" podCreationTimestamp="2026-06-03 13:53:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 13:53:49.684744838 +0000 UTC m=+5.629388806" watchObservedRunningTime="2026-06-03 13:53:49.685219135 +0000 UTC m=+5.629863090" Jun 03 13:53:50.145154 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:50.145117 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:50.145380 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:50.145275 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:53:50.145380 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:50.145358 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs podName:277e2d8b-f5ab-4f08-bb7e-b6981b3932fc nodeName:}" failed. No retries permitted until 2026-06-03 13:53:54.145321755 +0000 UTC m=+10.089965689 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs") pod "network-metrics-daemon-5m5l8" (UID: "277e2d8b-f5ab-4f08-bb7e-b6981b3932fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:53:50.246208 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:50.245563 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h84w6\" (UniqueName: \"kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6\") pod \"network-check-target-qsp8b\" (UID: \"8eb13e0d-051c-4a0e-a19d-6252080682a9\") " pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:50.246208 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:50.245750 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:53:50.246208 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:50.245774 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:53:50.246208 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:50.245786 2565 projected.go:194] Error preparing data for projected volume kube-api-access-h84w6 for pod openshift-network-diagnostics/network-check-target-qsp8b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:53:50.246208 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:50.245844 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6 podName:8eb13e0d-051c-4a0e-a19d-6252080682a9 nodeName:}" failed. No retries permitted until 2026-06-03 13:53:54.24582603 +0000 UTC m=+10.190469978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h84w6" (UniqueName: "kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6") pod "network-check-target-qsp8b" (UID: "8eb13e0d-051c-4a0e-a19d-6252080682a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:53:50.619046 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:50.618974 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:50.619330 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:50.619304 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:53:51.619442 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:51.619277 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:51.619442 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:51.619421 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:53:52.619711 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:52.619675 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:52.620174 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:52.619830 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:53:53.619053 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:53.618557 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:53.619053 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:53.618686 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:53:54.182092 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:54.182060 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:54.182548 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:54.182186 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:53:54.182548 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:54.182246 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs podName:277e2d8b-f5ab-4f08-bb7e-b6981b3932fc nodeName:}" failed. No retries permitted until 2026-06-03 13:54:02.182226685 +0000 UTC m=+18.126870619 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs") pod "network-metrics-daemon-5m5l8" (UID: "277e2d8b-f5ab-4f08-bb7e-b6981b3932fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:53:54.282544 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:54.282511 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h84w6\" (UniqueName: \"kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6\") pod \"network-check-target-qsp8b\" (UID: \"8eb13e0d-051c-4a0e-a19d-6252080682a9\") " pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:54.282727 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:54.282675 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:53:54.282727 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:54.282699 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:53:54.282727 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:54.282715 2565 projected.go:194] Error preparing data for projected volume kube-api-access-h84w6 for pod openshift-network-diagnostics/network-check-target-qsp8b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:53:54.282877 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:54.282780 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6 podName:8eb13e0d-051c-4a0e-a19d-6252080682a9 nodeName:}" failed. No retries permitted until 2026-06-03 13:54:02.282760841 +0000 UTC m=+18.227404774 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h84w6" (UniqueName: "kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6") pod "network-check-target-qsp8b" (UID: "8eb13e0d-051c-4a0e-a19d-6252080682a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:53:54.622105 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:54.622072 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:54.622265 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:54.622198 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:53:55.619667 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:55.619496 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:55.619667 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:55.619633 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:53:56.624509 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:56.624479 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:56.624960 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:56.624619 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:53:57.619544 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:57.619104 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:57.619544 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:57.619230 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:53:57.684456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:57.684410 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" event={"ID":"46b08401-ddce-482b-9907-33302b219ef2","Type":"ContainerStarted","Data":"19e636ee67c592ae5c97601815ba9da30bbe92532d5e230fbb77cd69ff57e1c7"} Jun 03 13:53:57.701188 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:57.701133 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-r4ntw" podStartSLOduration=3.809023293 podStartE2EDuration="13.701113645s" podCreationTimestamp="2026-06-03 13:53:44 +0000 UTC" firstStartedPulling="2026-06-03 13:53:47.589046795 +0000 UTC m=+3.533690732" lastFinishedPulling="2026-06-03 13:53:57.481137145 +0000 UTC m=+13.425781084" observedRunningTime="2026-06-03 13:53:57.700227013 +0000 UTC m=+13.644870969" watchObservedRunningTime="2026-06-03 13:53:57.701113645 +0000 UTC m=+13.645757602" Jun 03 13:53:58.618925 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:58.618746 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:53:58.619099 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:58.619037 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:53:58.687162 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:58.687123 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-np4q9" event={"ID":"77b40cee-f20f-4399-b4ff-42fc5f16335e","Type":"ContainerStarted","Data":"7aa1801cc0c9cf945793196d0841b74479501b83c07bfd66b8f231e1318bb542"} Jun 03 13:53:58.688554 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:58.688518 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6ns2r" event={"ID":"dedc278e-adec-4d58-bafb-047e9a36c84c","Type":"ContainerStarted","Data":"38cd576af7a8cd9ee4c258a2d1851920e815f318e5f3c381e3b405c0a469522c"} Jun 03 13:53:58.689915 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:58.689878 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gbtxm" event={"ID":"8c5d4af8-6b15-491f-9c16-884f366947e3","Type":"ContainerStarted","Data":"43cbb5d0f68a0d8de4d498c6b522bf5173c115036f0eab3d574dfadac9ddeefe"} Jun 03 13:53:58.691259 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:58.691234 2565 generic.go:358] "Generic (PLEG): container finished" podID="1d997ce2-6758-4937-8e63-dad6bf7b689a" containerID="24c1cde9efe46fc9d29e24c34de349f652751c762dab975aed09931bdf5a3362" exitCode=0 Jun 03 13:53:58.691367 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:58.691306 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f5chv" event={"ID":"1d997ce2-6758-4937-8e63-dad6bf7b689a","Type":"ContainerDied","Data":"24c1cde9efe46fc9d29e24c34de349f652751c762dab975aed09931bdf5a3362"} Jun 03 13:53:58.692666 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:58.692613 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" event={"ID":"cc27edab-d11b-4e37-a749-9a7ca106e24e","Type":"ContainerStarted","Data":"96e51a5581b4591fade9534c8ceab61b01eaa8e530423bd161f0263aacf3935b"} Jun 03 13:53:58.702804 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:58.702753 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-np4q9" podStartSLOduration=4.865765242 podStartE2EDuration="14.7027369s" podCreationTimestamp="2026-06-03 13:53:44 +0000 UTC" firstStartedPulling="2026-06-03 13:53:47.587150189 +0000 UTC m=+3.531794136" lastFinishedPulling="2026-06-03 13:53:57.424121844 +0000 UTC m=+13.368765794" observedRunningTime="2026-06-03 13:53:58.70185264 +0000 UTC m=+14.646496597" watchObservedRunningTime="2026-06-03 13:53:58.7027369 +0000 UTC m=+14.647380855" Jun 03 13:53:58.717448 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:58.717379 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-gbtxm" podStartSLOduration=4.86997888 podStartE2EDuration="14.717363894s" podCreationTimestamp="2026-06-03 13:53:44 +0000 UTC" firstStartedPulling="2026-06-03 13:53:47.57969293 +0000 UTC m=+3.524336869" lastFinishedPulling="2026-06-03 13:53:57.427077943 +0000 UTC m=+13.371721883" observedRunningTime="2026-06-03 13:53:58.716866501 +0000 UTC m=+14.661510457" watchObservedRunningTime="2026-06-03 13:53:58.717363894 +0000 UTC m=+14.662007850" Jun 03 13:53:58.729156 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:58.729121 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6ns2r" podStartSLOduration=4.886869088 podStartE2EDuration="14.729109988s" podCreationTimestamp="2026-06-03 13:53:44 +0000 UTC" firstStartedPulling="2026-06-03 13:53:47.580535619 +0000 UTC m=+3.525179552" lastFinishedPulling="2026-06-03 13:53:57.422776516 +0000 UTC m=+13.367420452" observedRunningTime="2026-06-03 13:53:58.728925282 +0000 UTC m=+14.673569237" watchObservedRunningTime="2026-06-03 13:53:58.729109988 +0000 UTC m=+14.673753942" Jun 03 13:53:59.619264 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:59.619231 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:53:59.619441 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:59.619372 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:53:59.689313 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:59.689282 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-nsnhl"] Jun 03 13:53:59.692488 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:59.692465 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:53:59.692590 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:59.692543 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsnhl" podUID="561a2ef6-f963-4946-bdf9-8336eaae20d6" Jun 03 13:53:59.695371 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:59.695314 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nbbzd" event={"ID":"4b8242da-a838-4502-88d1-ecc03b3386ce","Type":"ContainerStarted","Data":"34782ef7319e5f71162759c033cabb6fd9913b833d092dccb769c604b02bb1ff"} Jun 03 13:53:59.727415 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:59.727382 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/561a2ef6-f963-4946-bdf9-8336eaae20d6-dbus\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:53:59.727522 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:59.727445 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/561a2ef6-f963-4946-bdf9-8336eaae20d6-kubelet-config\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:53:59.727522 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:59.727474 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:53:59.731011 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:59.730966 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nbbzd" podStartSLOduration=5.892937794 podStartE2EDuration="15.730954041s" podCreationTimestamp="2026-06-03 13:53:44 +0000 UTC" firstStartedPulling="2026-06-03 13:53:47.584820243 +0000 UTC m=+3.529464179" lastFinishedPulling="2026-06-03 13:53:57.422836485 +0000 UTC m=+13.367480426" observedRunningTime="2026-06-03 13:53:59.73060187 +0000 UTC m=+15.675245826" watchObservedRunningTime="2026-06-03 13:53:59.730954041 +0000 UTC m=+15.675597996" Jun 03 13:53:59.828011 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:59.827986 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/561a2ef6-f963-4946-bdf9-8336eaae20d6-kubelet-config\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:53:59.828155 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:59.828031 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:53:59.828155 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:59.828105 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/561a2ef6-f963-4946-bdf9-8336eaae20d6-dbus\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:53:59.828155 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:59.828114 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/561a2ef6-f963-4946-bdf9-8336eaae20d6-kubelet-config\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:53:59.828566 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:59.828535 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 13:53:59.828566 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:53:59.828553 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/561a2ef6-f963-4946-bdf9-8336eaae20d6-dbus\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:53:59.828752 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:53:59.828602 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret podName:561a2ef6-f963-4946-bdf9-8336eaae20d6 nodeName:}" failed. No retries permitted until 2026-06-03 13:54:00.32858123 +0000 UTC m=+16.273225180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret") pod "global-pull-secret-syncer-nsnhl" (UID: "561a2ef6-f963-4946-bdf9-8336eaae20d6") : object "kube-system"/"original-pull-secret" not registered Jun 03 13:54:00.332581 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:00.332550 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:00.332755 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:00.332677 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 13:54:00.332755 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:00.332740 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret podName:561a2ef6-f963-4946-bdf9-8336eaae20d6 nodeName:}" failed. No retries permitted until 2026-06-03 13:54:01.332723457 +0000 UTC m=+17.277367397 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret") pod "global-pull-secret-syncer-nsnhl" (UID: "561a2ef6-f963-4946-bdf9-8336eaae20d6") : object "kube-system"/"original-pull-secret" not registered Jun 03 13:54:00.618852 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:00.618769 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:00.619015 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:00.618887 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:54:00.640477 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:00.640441 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-np4q9" Jun 03 13:54:00.641060 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:00.641034 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-np4q9" Jun 03 13:54:00.698598 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:00.698569 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-np4q9" Jun 03 13:54:00.699257 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:00.699134 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-np4q9" Jun 03 13:54:01.340357 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:01.340309 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:01.340529 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:01.340459 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 13:54:01.340529 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:01.340508 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret podName:561a2ef6-f963-4946-bdf9-8336eaae20d6 nodeName:}" failed. No retries permitted until 2026-06-03 13:54:03.340494663 +0000 UTC m=+19.285138596 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret") pod "global-pull-secret-syncer-nsnhl" (UID: "561a2ef6-f963-4946-bdf9-8336eaae20d6") : object "kube-system"/"original-pull-secret" not registered Jun 03 13:54:01.619138 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:01.619057 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:01.619305 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:01.619061 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:01.619305 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:01.619188 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsnhl" podUID="561a2ef6-f963-4946-bdf9-8336eaae20d6" Jun 03 13:54:01.619305 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:01.619273 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:54:02.246167 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:02.246141 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:02.246799 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:02.246288 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:54:02.246799 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:02.246374 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs podName:277e2d8b-f5ab-4f08-bb7e-b6981b3932fc nodeName:}" failed. No retries permitted until 2026-06-03 13:54:18.246353688 +0000 UTC m=+34.190997638 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs") pod "network-metrics-daemon-5m5l8" (UID: "277e2d8b-f5ab-4f08-bb7e-b6981b3932fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:54:02.346608 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:02.346572 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h84w6\" (UniqueName: \"kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6\") pod \"network-check-target-qsp8b\" (UID: \"8eb13e0d-051c-4a0e-a19d-6252080682a9\") " pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:02.346853 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:02.346744 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:54:02.346853 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:02.346763 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:54:02.346853 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:02.346773 2565 projected.go:194] Error preparing data for projected volume kube-api-access-h84w6 for pod openshift-network-diagnostics/network-check-target-qsp8b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:54:02.346853 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:02.346831 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6 podName:8eb13e0d-051c-4a0e-a19d-6252080682a9 nodeName:}" failed. No retries permitted until 2026-06-03 13:54:18.346813637 +0000 UTC m=+34.291457570 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h84w6" (UniqueName: "kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6") pod "network-check-target-qsp8b" (UID: "8eb13e0d-051c-4a0e-a19d-6252080682a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:54:02.619359 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:02.619277 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:02.619502 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:02.619445 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:54:03.354037 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:03.353996 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:03.354504 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:03.354149 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 13:54:03.354504 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:03.354220 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret podName:561a2ef6-f963-4946-bdf9-8336eaae20d6 nodeName:}" failed. No retries permitted until 2026-06-03 13:54:07.354200403 +0000 UTC m=+23.298844341 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret") pod "global-pull-secret-syncer-nsnhl" (UID: "561a2ef6-f963-4946-bdf9-8336eaae20d6") : object "kube-system"/"original-pull-secret" not registered Jun 03 13:54:03.619554 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:03.619474 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:03.619705 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:03.619474 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:03.619705 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:03.619613 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:54:03.619814 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:03.619733 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsnhl" podUID="561a2ef6-f963-4946-bdf9-8336eaae20d6" Jun 03 13:54:04.619670 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:04.619633 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:04.620063 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:04.619750 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:54:05.618828 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:05.618797 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:05.618962 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:05.618797 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:05.618962 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:05.618892 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsnhl" podUID="561a2ef6-f963-4946-bdf9-8336eaae20d6" Jun 03 13:54:05.619035 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:05.618957 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:54:06.519706 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:06.519448 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 03 13:54:06.566747 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:06.566622 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-03T13:54:06.519466863Z","UUID":"e472c96c-474b-48ee-95b2-a8a0e74f1d50","Handler":null,"Name":"","Endpoint":""} Jun 03 13:54:06.568712 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:06.568693 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 03 13:54:06.568806 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:06.568724 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 03 13:54:06.619203 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:06.619183 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:06.619294 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:06.619278 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:54:06.711514 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:06.711483 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" event={"ID":"d49f3147-aee7-4a57-88f6-ad00854e9eb3","Type":"ContainerStarted","Data":"d5921da956bd1caa399e3ce85caad24d1fe3424db2f4d7b90b2ce74782490cdc"} Jun 03 13:54:06.711666 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:06.711523 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" event={"ID":"d49f3147-aee7-4a57-88f6-ad00854e9eb3","Type":"ContainerStarted","Data":"f7645e645cefd3ea7799f9ed29a5159b9617c2cef92760151d0d23aa7823e90f"} Jun 03 13:54:06.711666 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:06.711537 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" event={"ID":"d49f3147-aee7-4a57-88f6-ad00854e9eb3","Type":"ContainerStarted","Data":"5350dba92610ebea707ba46c25c33e3df246c44c6708a9b4bb2033e5b76c5ea4"} Jun 03 13:54:06.711666 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:06.711550 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" event={"ID":"d49f3147-aee7-4a57-88f6-ad00854e9eb3","Type":"ContainerStarted","Data":"29eedd997497a3ad471e2e5e58bcacfe6b2395e6352f00dfdd4472ba628d4841"} Jun 03 13:54:06.711666 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:06.711562 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" event={"ID":"d49f3147-aee7-4a57-88f6-ad00854e9eb3","Type":"ContainerStarted","Data":"b08971560590612f5955cc46315079b808514aefbae3c8b207bb9c4bfa9e4d1f"} Jun 03 13:54:06.712788 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:06.712764 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zp6hm" event={"ID":"4df1db34-2741-48c1-844e-231fbafcd068","Type":"ContainerStarted","Data":"31240f4a3f99e60b0f381f625e3259ea3a5ef1cd2659e65dd472c85f937a3dab"} Jun 03 13:54:06.714373 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:06.714331 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" event={"ID":"cc27edab-d11b-4e37-a749-9a7ca106e24e","Type":"ContainerStarted","Data":"6252fb6bda90e952ba19ab50b948be86dd9d65ee4afee86adf2b3563c5a7b11d"} Jun 03 13:54:06.747481 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:06.747444 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-zp6hm" podStartSLOduration=4.168306721 podStartE2EDuration="22.747434153s" podCreationTimestamp="2026-06-03 13:53:44 +0000 UTC" firstStartedPulling="2026-06-03 13:53:47.582768177 +0000 UTC m=+3.527412109" lastFinishedPulling="2026-06-03 13:54:06.161895605 +0000 UTC m=+22.106539541" observedRunningTime="2026-06-03 13:54:06.74736437 +0000 UTC m=+22.692008325" watchObservedRunningTime="2026-06-03 13:54:06.747434153 +0000 UTC m=+22.692078110" Jun 03 13:54:07.385322 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:07.385289 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:07.385523 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:07.385428 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 13:54:07.385523 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:07.385491 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret podName:561a2ef6-f963-4946-bdf9-8336eaae20d6 nodeName:}" failed. No retries permitted until 2026-06-03 13:54:15.385470292 +0000 UTC m=+31.330114224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret") pod "global-pull-secret-syncer-nsnhl" (UID: "561a2ef6-f963-4946-bdf9-8336eaae20d6") : object "kube-system"/"original-pull-secret" not registered Jun 03 13:54:07.619213 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:07.619184 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:07.619760 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:07.619184 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:07.619760 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:07.619299 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsnhl" podUID="561a2ef6-f963-4946-bdf9-8336eaae20d6" Jun 03 13:54:07.619760 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:07.619367 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:54:07.719432 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:07.719404 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" event={"ID":"d49f3147-aee7-4a57-88f6-ad00854e9eb3","Type":"ContainerStarted","Data":"d28fbe092626985a43abdcf428a83c28956a46c2f3d2bef478dab937b6924618"} Jun 03 13:54:08.619426 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:08.619392 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:08.619920 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:08.619539 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:54:08.723137 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:08.723101 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" event={"ID":"cc27edab-d11b-4e37-a749-9a7ca106e24e","Type":"ContainerStarted","Data":"d6d5bc69d2f0690974a2c3ed285a4ec0462acbb1978b380e1d233f828938f3e9"} Jun 03 13:54:08.742438 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:08.742396 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2l7qx" podStartSLOduration=4.607797216 podStartE2EDuration="24.742382244s" podCreationTimestamp="2026-06-03 13:53:44 +0000 UTC" firstStartedPulling="2026-06-03 13:53:47.577441399 +0000 UTC m=+3.522085332" lastFinishedPulling="2026-06-03 13:54:07.712026425 +0000 UTC m=+23.656670360" observedRunningTime="2026-06-03 13:54:08.742164116 +0000 UTC m=+24.686808069" watchObservedRunningTime="2026-06-03 13:54:08.742382244 +0000 UTC m=+24.687026198" Jun 03 13:54:09.618650 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:09.618613 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:09.618816 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:09.618740 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:54:09.618816 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:09.618802 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:09.618949 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:09.618925 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsnhl" podUID="561a2ef6-f963-4946-bdf9-8336eaae20d6" Jun 03 13:54:09.727771 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:09.727729 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" event={"ID":"d49f3147-aee7-4a57-88f6-ad00854e9eb3","Type":"ContainerStarted","Data":"06341640b3978547e9374958d63b51d0b7c5552c5ee6d19dac47963584b03556"} Jun 03 13:54:10.619610 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:10.619577 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:10.619790 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:10.619729 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:54:11.619741 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:11.619563 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:11.620272 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:11.619563 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:11.620272 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:11.619814 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:54:11.620272 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:11.619879 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsnhl" podUID="561a2ef6-f963-4946-bdf9-8336eaae20d6" Jun 03 13:54:11.735451 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:11.735419 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" event={"ID":"d49f3147-aee7-4a57-88f6-ad00854e9eb3","Type":"ContainerStarted","Data":"703709a2d810dc2ccf9280ec925ca4da7f8a242e4ec5401658007b9c867259fc"} Jun 03 13:54:11.735767 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:11.735742 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:54:11.735887 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:11.735774 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:54:11.737144 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:11.737123 2565 generic.go:358] "Generic (PLEG): container finished" podID="1d997ce2-6758-4937-8e63-dad6bf7b689a" containerID="081012bba308eec047720995383f2eed8c8da9e5fe410ca583564ed9ee696d20" exitCode=0 Jun 03 13:54:11.737247 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:11.737154 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f5chv" event={"ID":"1d997ce2-6758-4937-8e63-dad6bf7b689a","Type":"ContainerDied","Data":"081012bba308eec047720995383f2eed8c8da9e5fe410ca583564ed9ee696d20"} Jun 03 13:54:11.751518 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:11.751499 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:54:11.762161 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:11.762126 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" podStartSLOduration=9.236969589 podStartE2EDuration="27.762115697s" podCreationTimestamp="2026-06-03 13:53:44 +0000 UTC" firstStartedPulling="2026-06-03 13:53:47.583753578 +0000 UTC m=+3.528397514" lastFinishedPulling="2026-06-03 13:54:06.108899677 +0000 UTC m=+22.053543622" observedRunningTime="2026-06-03 13:54:11.76179689 +0000 UTC m=+27.706440845" watchObservedRunningTime="2026-06-03 13:54:11.762115697 +0000 UTC m=+27.706759652" Jun 03 13:54:12.619059 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:12.619034 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:12.619166 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:12.619144 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:54:12.741374 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:12.741322 2565 generic.go:358] "Generic (PLEG): container finished" podID="1d997ce2-6758-4937-8e63-dad6bf7b689a" containerID="af06c0dbe4ffc8329e0247688de4881f7d2bda5f9dbf62931bbd60e9b74589f6" exitCode=0 Jun 03 13:54:12.741794 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:12.741374 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f5chv" event={"ID":"1d997ce2-6758-4937-8e63-dad6bf7b689a","Type":"ContainerDied","Data":"af06c0dbe4ffc8329e0247688de4881f7d2bda5f9dbf62931bbd60e9b74589f6"} Jun 03 13:54:12.742647 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:12.742145 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:54:12.756467 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:12.756410 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:54:12.940012 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:12.939986 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5m5l8"] Jun 03 13:54:12.940135 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:12.940087 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:12.940207 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:12.940188 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:54:12.943264 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:12.943240 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nsnhl"] Jun 03 13:54:12.943396 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:12.943357 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:12.943491 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:12.943471 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsnhl" podUID="561a2ef6-f963-4946-bdf9-8336eaae20d6" Jun 03 13:54:12.943903 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:12.943884 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qsp8b"] Jun 03 13:54:12.944008 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:12.943961 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:12.944060 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:12.944016 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:54:13.745747 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:13.745658 2565 generic.go:358] "Generic (PLEG): container finished" podID="1d997ce2-6758-4937-8e63-dad6bf7b689a" containerID="ca335232458c512e2c624bff3befc4474d1d8f9987316417ec58f8f4dd230d55" exitCode=0 Jun 03 13:54:13.745747 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:13.745731 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f5chv" event={"ID":"1d997ce2-6758-4937-8e63-dad6bf7b689a","Type":"ContainerDied","Data":"ca335232458c512e2c624bff3befc4474d1d8f9987316417ec58f8f4dd230d55"} Jun 03 13:54:14.619677 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:14.619638 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:14.619874 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:14.619726 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsnhl" podUID="561a2ef6-f963-4946-bdf9-8336eaae20d6" Jun 03 13:54:14.619874 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:14.619818 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:14.620049 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:14.619928 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:14.620049 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:14.619947 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:54:14.620049 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:14.620009 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:54:15.444364 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:15.444283 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:15.444703 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:15.444462 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 13:54:15.444703 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:15.444545 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret podName:561a2ef6-f963-4946-bdf9-8336eaae20d6 nodeName:}" failed. No retries permitted until 2026-06-03 13:54:31.444522418 +0000 UTC m=+47.389166351 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret") pod "global-pull-secret-syncer-nsnhl" (UID: "561a2ef6-f963-4946-bdf9-8336eaae20d6") : object "kube-system"/"original-pull-secret" not registered Jun 03 13:54:16.618871 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:16.618835 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:16.618871 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:16.618864 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:16.619378 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:16.618960 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:54:16.619378 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:16.619000 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:16.619378 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:16.619099 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:54:16.619378 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:16.619142 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsnhl" podUID="561a2ef6-f963-4946-bdf9-8336eaae20d6" Jun 03 13:54:18.267151 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.266910 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:18.267767 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:18.267078 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:54:18.267767 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:18.267317 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs podName:277e2d8b-f5ab-4f08-bb7e-b6981b3932fc nodeName:}" failed. No retries permitted until 2026-06-03 13:54:50.267294524 +0000 UTC m=+66.211938480 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs") pod "network-metrics-daemon-5m5l8" (UID: "277e2d8b-f5ab-4f08-bb7e-b6981b3932fc") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 13:54:18.367796 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.367762 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h84w6\" (UniqueName: \"kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6\") pod \"network-check-target-qsp8b\" (UID: \"8eb13e0d-051c-4a0e-a19d-6252080682a9\") " pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:18.367981 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:18.367957 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 13:54:18.368034 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:18.367983 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 13:54:18.368034 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:18.367996 2565 projected.go:194] Error preparing data for projected volume kube-api-access-h84w6 for pod openshift-network-diagnostics/network-check-target-qsp8b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:54:18.368106 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:18.368064 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6 podName:8eb13e0d-051c-4a0e-a19d-6252080682a9 nodeName:}" failed. No retries permitted until 2026-06-03 13:54:50.368045659 +0000 UTC m=+66.312689594 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-h84w6" (UniqueName: "kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6") pod "network-check-target-qsp8b" (UID: "8eb13e0d-051c-4a0e-a19d-6252080682a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 13:54:18.619019 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.618944 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:18.619161 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.619081 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:18.619161 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:18.619093 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:54:18.619279 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:18.619210 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nsnhl" podUID="561a2ef6-f963-4946-bdf9-8336eaae20d6" Jun 03 13:54:18.619279 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.619255 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:18.619395 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:18.619327 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qsp8b" podUID="8eb13e0d-051c-4a0e-a19d-6252080682a9" Jun 03 13:54:18.889063 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.888984 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-89.ec2.internal" event="NodeReady" Jun 03 13:54:18.889229 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.889126 2565 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 03 13:54:18.927859 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.927827 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz"] Jun 03 13:54:18.935674 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.935649 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cd7686cb6-7bhjc"] Jun 03 13:54:18.935858 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.935829 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:54:18.937656 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.937637 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv"] Jun 03 13:54:18.937809 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.937785 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:18.938482 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.938459 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 03 13:54:18.938482 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.938472 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7fx7r\"" Jun 03 13:54:18.938709 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.938696 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 03 13:54:18.940799 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.940778 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4"] Jun 03 13:54:18.940919 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.940904 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:18.941209 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.941038 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 03 13:54:18.941564 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.941547 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 03 13:54:18.941756 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.941725 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wvjmg\"" Jun 03 13:54:18.941913 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.941895 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 03 13:54:18.947290 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.946310 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l"] Jun 03 13:54:18.947290 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.946586 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:54:18.947290 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.946945 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 03 13:54:18.947974 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.947952 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 03 13:54:18.948058 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.948022 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 03 13:54:18.948058 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.948030 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 03 13:54:18.948212 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.948193 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 03 13:54:18.948373 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.948203 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 03 13:54:18.948478 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.948456 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 03 13:54:18.951211 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.950855 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" Jun 03 13:54:18.951904 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.951872 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 03 13:54:18.952931 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.952685 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz"] Jun 03 13:54:18.953535 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.953518 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4"] Jun 03 13:54:18.956750 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.955977 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7d2wt"] Jun 03 13:54:18.959096 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.958845 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-zn8wp\"" Jun 03 13:54:18.960140 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.959254 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 03 13:54:18.960140 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.959629 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 03 13:54:18.960812 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.960796 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cd7686cb6-7bhjc"] Jun 03 13:54:18.960812 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.960814 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l"] Jun 03 13:54:18.960917 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.960823 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv"] Jun 03 13:54:18.960952 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.960937 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:18.963558 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.963364 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 03 13:54:18.963558 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.963397 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 03 13:54:18.963558 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.963473 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zsshc\"" Jun 03 13:54:18.994662 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:18.994636 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7d2wt"] Jun 03 13:54:19.056826 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.056791 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p66h6"] Jun 03 13:54:19.062455 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.062432 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:54:19.065097 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.065057 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 03 13:54:19.065097 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.065062 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 03 13:54:19.065287 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.065150 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gxcmr\"" Jun 03 13:54:19.066045 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.066027 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 03 13:54:19.072063 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072039 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a45139c3-0352-430c-acdc-6fe24607c5fc-ca\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.072156 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072093 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-bound-sa-token\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.072156 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072115 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/611161c3-524f-4088-8e3f-67d41f5350da-tmp\") pod \"klusterlet-addon-workmgr-5fc44ffdf6-gj2q4\" (UID: \"611161c3-524f-4088-8e3f-67d41f5350da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:54:19.072237 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072162 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/a45139c3-0352-430c-acdc-6fe24607c5fc-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.072295 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072233 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-ca-trust-extracted\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.072295 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072260 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-trusted-ca\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.072387 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072314 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:54:19.072434 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072402 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-installation-pull-secrets\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.072473 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072452 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6724d1a1-68ea-4f1d-a782-7bbaf376213c-config-volume\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:19.072564 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072480 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/611161c3-524f-4088-8e3f-67d41f5350da-klusterlet-config\") pod \"klusterlet-addon-workmgr-5fc44ffdf6-gj2q4\" (UID: \"611161c3-524f-4088-8e3f-67d41f5350da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:54:19.072564 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072519 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6724d1a1-68ea-4f1d-a782-7bbaf376213c-tmp-dir\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:19.072564 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072545 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:54:19.072690 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072571 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dad994ec-200d-446b-8078-99e71623f5e5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l\" (UID: \"dad994ec-200d-446b-8078-99e71623f5e5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" Jun 03 13:54:19.072690 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072599 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a45139c3-0352-430c-acdc-6fe24607c5fc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.072690 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072640 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z529n\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-kube-api-access-z529n\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.072690 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072683 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a45139c3-0352-430c-acdc-6fe24607c5fc-hub\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.072933 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072711 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j4tq\" (UniqueName: \"kubernetes.io/projected/a45139c3-0352-430c-acdc-6fe24607c5fc-kube-api-access-4j4tq\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.072933 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072731 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.072933 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072754 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsr2p\" (UniqueName: \"kubernetes.io/projected/dad994ec-200d-446b-8078-99e71623f5e5-kube-api-access-qsr2p\") pod \"managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l\" (UID: \"dad994ec-200d-446b-8078-99e71623f5e5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" Jun 03 13:54:19.072933 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072778 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a45139c3-0352-430c-acdc-6fe24607c5fc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.072933 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072819 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:19.072933 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072840 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv4rc\" (UniqueName: \"kubernetes.io/projected/6724d1a1-68ea-4f1d-a782-7bbaf376213c-kube-api-access-fv4rc\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:19.072933 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072862 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-image-registry-private-configuration\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.072933 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072886 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-certificates\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.072933 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.072911 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jct6\" (UniqueName: \"kubernetes.io/projected/611161c3-524f-4088-8e3f-67d41f5350da-kube-api-access-5jct6\") pod \"klusterlet-addon-workmgr-5fc44ffdf6-gj2q4\" (UID: \"611161c3-524f-4088-8e3f-67d41f5350da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:54:19.076701 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.076668 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p66h6"] Jun 03 13:54:19.173354 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173261 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-ca-trust-extracted\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.173354 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173302 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:54:19.173616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173370 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6724d1a1-68ea-4f1d-a782-7bbaf376213c-tmp-dir\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:19.173616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173397 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dad994ec-200d-446b-8078-99e71623f5e5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l\" (UID: \"dad994ec-200d-446b-8078-99e71623f5e5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" Jun 03 13:54:19.173616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173425 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4j4tq\" (UniqueName: \"kubernetes.io/projected/a45139c3-0352-430c-acdc-6fe24607c5fc-kube-api-access-4j4tq\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.173616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173446 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a45139c3-0352-430c-acdc-6fe24607c5fc-hub\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.173616 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.173465 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 03 13:54:19.173616 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.173540 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert podName:4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b nodeName:}" failed. No retries permitted until 2026-06-03 13:54:19.673520545 +0000 UTC m=+35.618164491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-9qlnz" (UID: "4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b") : secret "networking-console-plugin-cert" not found Jun 03 13:54:19.173881 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173727 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-ca-trust-extracted\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.173881 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173748 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a45139c3-0352-430c-acdc-6fe24607c5fc-ca\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.173881 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173784 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fv4rc\" (UniqueName: \"kubernetes.io/projected/6724d1a1-68ea-4f1d-a782-7bbaf376213c-kube-api-access-fv4rc\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:19.173881 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173804 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6724d1a1-68ea-4f1d-a782-7bbaf376213c-tmp-dir\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:19.173881 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173814 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:54:19.173881 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173876 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/a45139c3-0352-430c-acdc-6fe24607c5fc-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.174080 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173902 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/611161c3-524f-4088-8e3f-67d41f5350da-tmp\") pod \"klusterlet-addon-workmgr-5fc44ffdf6-gj2q4\" (UID: \"611161c3-524f-4088-8e3f-67d41f5350da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:54:19.174080 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173929 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-trusted-ca\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.174080 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173953 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:54:19.174080 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.173989 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-installation-pull-secrets\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.174080 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.174014 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/611161c3-524f-4088-8e3f-67d41f5350da-klusterlet-config\") pod \"klusterlet-addon-workmgr-5fc44ffdf6-gj2q4\" (UID: \"611161c3-524f-4088-8e3f-67d41f5350da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:54:19.174080 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.174040 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6724d1a1-68ea-4f1d-a782-7bbaf376213c-config-volume\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:19.174080 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.174068 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a45139c3-0352-430c-acdc-6fe24607c5fc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.174923 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.174887 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z529n\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-kube-api-access-z529n\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.176756 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.175729 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6724d1a1-68ea-4f1d-a782-7bbaf376213c-config-volume\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:19.176756 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.175795 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.176756 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.175832 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qsr2p\" (UniqueName: \"kubernetes.io/projected/dad994ec-200d-446b-8078-99e71623f5e5-kube-api-access-qsr2p\") pod \"managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l\" (UID: \"dad994ec-200d-446b-8078-99e71623f5e5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" Jun 03 13:54:19.176756 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.175877 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:19.176756 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.175920 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trm64\" (UniqueName: \"kubernetes.io/projected/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-kube-api-access-trm64\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:54:19.176756 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.175948 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:54:19.176756 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.175960 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5jct6\" (UniqueName: \"kubernetes.io/projected/611161c3-524f-4088-8e3f-67d41f5350da-kube-api-access-5jct6\") pod \"klusterlet-addon-workmgr-5fc44ffdf6-gj2q4\" (UID: \"611161c3-524f-4088-8e3f-67d41f5350da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:54:19.176756 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.176001 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-image-registry-private-configuration\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.176756 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.176031 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-certificates\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.176756 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.176084 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a45139c3-0352-430c-acdc-6fe24607c5fc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.176756 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.176107 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/611161c3-524f-4088-8e3f-67d41f5350da-tmp\") pod \"klusterlet-addon-workmgr-5fc44ffdf6-gj2q4\" (UID: \"611161c3-524f-4088-8e3f-67d41f5350da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:54:19.176756 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.176132 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-bound-sa-token\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.177410 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.177035 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-trusted-ca\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.177649 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.177625 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:54:19.177649 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.177647 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cd7686cb6-7bhjc: secret "image-registry-tls" not found Jun 03 13:54:19.177807 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.177723 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls podName:063b7d6f-5926-4d42-a29c-3ed54eaeb18a nodeName:}" failed. No retries permitted until 2026-06-03 13:54:19.67770017 +0000 UTC m=+35.622344103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls") pod "image-registry-7cd7686cb6-7bhjc" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a") : secret "image-registry-tls" not found Jun 03 13:54:19.178910 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.178254 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:54:19.178910 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.178527 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls podName:6724d1a1-68ea-4f1d-a782-7bbaf376213c nodeName:}" failed. No retries permitted until 2026-06-03 13:54:19.678499959 +0000 UTC m=+35.623143903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls") pod "dns-default-7d2wt" (UID: "6724d1a1-68ea-4f1d-a782-7bbaf376213c") : secret "dns-default-metrics-tls" not found Jun 03 13:54:19.179164 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.179114 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-certificates\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.180328 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.179738 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/a45139c3-0352-430c-acdc-6fe24607c5fc-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.180470 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.180445 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-image-registry-private-configuration\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.180615 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.180592 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a45139c3-0352-430c-acdc-6fe24607c5fc-ca\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.180677 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.180600 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/611161c3-524f-4088-8e3f-67d41f5350da-klusterlet-config\") pod \"klusterlet-addon-workmgr-5fc44ffdf6-gj2q4\" (UID: \"611161c3-524f-4088-8e3f-67d41f5350da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:54:19.181081 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.181053 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dad994ec-200d-446b-8078-99e71623f5e5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l\" (UID: \"dad994ec-200d-446b-8078-99e71623f5e5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" Jun 03 13:54:19.181155 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.181080 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a45139c3-0352-430c-acdc-6fe24607c5fc-hub\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.181642 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.181618 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-installation-pull-secrets\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.183269 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.183138 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv4rc\" (UniqueName: \"kubernetes.io/projected/6724d1a1-68ea-4f1d-a782-7bbaf376213c-kube-api-access-fv4rc\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:19.184925 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.183756 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a45139c3-0352-430c-acdc-6fe24607c5fc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.184925 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.183954 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j4tq\" (UniqueName: \"kubernetes.io/projected/a45139c3-0352-430c-acdc-6fe24607c5fc-kube-api-access-4j4tq\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.184925 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.184807 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a45139c3-0352-430c-acdc-6fe24607c5fc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f786849c7-mjzgv\" (UID: \"a45139c3-0352-430c-acdc-6fe24607c5fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.186919 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.186894 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z529n\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-kube-api-access-z529n\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.187133 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.187106 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsr2p\" (UniqueName: \"kubernetes.io/projected/dad994ec-200d-446b-8078-99e71623f5e5-kube-api-access-qsr2p\") pod \"managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l\" (UID: \"dad994ec-200d-446b-8078-99e71623f5e5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" Jun 03 13:54:19.187372 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.187351 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-bound-sa-token\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.211994 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.211969 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jct6\" (UniqueName: \"kubernetes.io/projected/611161c3-524f-4088-8e3f-67d41f5350da-kube-api-access-5jct6\") pod \"klusterlet-addon-workmgr-5fc44ffdf6-gj2q4\" (UID: \"611161c3-524f-4088-8e3f-67d41f5350da\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:54:19.277433 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.277398 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:54:19.278059 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.277518 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:54:19.278059 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.277520 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-trm64\" (UniqueName: \"kubernetes.io/projected/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-kube-api-access-trm64\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:54:19.278059 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.277526 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:54:19.278059 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.277743 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert podName:04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea nodeName:}" failed. No retries permitted until 2026-06-03 13:54:19.777717544 +0000 UTC m=+35.722361482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert") pod "ingress-canary-p66h6" (UID: "04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea") : secret "canary-serving-cert" not found Jun 03 13:54:19.286222 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.286204 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:54:19.289703 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.289683 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trm64\" (UniqueName: \"kubernetes.io/projected/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-kube-api-access-trm64\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:54:19.295442 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.295426 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" Jun 03 13:54:19.681661 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.681602 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:54:19.681827 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.681755 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:19.681827 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.681766 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 03 13:54:19.681827 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.681791 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:19.681946 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.681836 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert podName:4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b nodeName:}" failed. No retries permitted until 2026-06-03 13:54:20.681815216 +0000 UTC m=+36.626459156 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-9qlnz" (UID: "4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b") : secret "networking-console-plugin-cert" not found Jun 03 13:54:19.681946 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.681872 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:54:19.681946 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.681889 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:54:19.681946 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.681893 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cd7686cb6-7bhjc: secret "image-registry-tls" not found Jun 03 13:54:19.682096 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.681948 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls podName:063b7d6f-5926-4d42-a29c-3ed54eaeb18a nodeName:}" failed. No retries permitted until 2026-06-03 13:54:20.681930943 +0000 UTC m=+36.626574884 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls") pod "image-registry-7cd7686cb6-7bhjc" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a") : secret "image-registry-tls" not found Jun 03 13:54:19.682096 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.681966 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls podName:6724d1a1-68ea-4f1d-a782-7bbaf376213c nodeName:}" failed. No retries permitted until 2026-06-03 13:54:20.68195869 +0000 UTC m=+36.626602622 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls") pod "dns-default-7d2wt" (UID: "6724d1a1-68ea-4f1d-a782-7bbaf376213c") : secret "dns-default-metrics-tls" not found Jun 03 13:54:19.782973 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:19.782935 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:54:19.783129 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.783078 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:54:19.783182 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:19.783155 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert podName:04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea nodeName:}" failed. No retries permitted until 2026-06-03 13:54:20.783136986 +0000 UTC m=+36.727780926 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert") pod "ingress-canary-p66h6" (UID: "04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea") : secret "canary-serving-cert" not found Jun 03 13:54:20.433285 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.433146 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4"] Jun 03 13:54:20.435514 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.435492 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l"] Jun 03 13:54:20.438477 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.438455 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv"] Jun 03 13:54:20.466082 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:54:20.466051 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611161c3_524f_4088_8e3f_67d41f5350da.slice/crio-176e6f71dae1a621fe6f1835ea9a4064b570086291f024fce2b54c6ae4ccc82c WatchSource:0}: Error finding container 176e6f71dae1a621fe6f1835ea9a4064b570086291f024fce2b54c6ae4ccc82c: Status 404 returned error can't find the container with id 176e6f71dae1a621fe6f1835ea9a4064b570086291f024fce2b54c6ae4ccc82c Jun 03 13:54:20.466465 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:54:20.466443 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddad994ec_200d_446b_8078_99e71623f5e5.slice/crio-4b2d48a34c82f6c8658f8d14ca0483b973c656b3ede0bbe050f2edce7122e72e WatchSource:0}: Error finding container 4b2d48a34c82f6c8658f8d14ca0483b973c656b3ede0bbe050f2edce7122e72e: Status 404 returned error can't find the container with id 4b2d48a34c82f6c8658f8d14ca0483b973c656b3ede0bbe050f2edce7122e72e Jun 03 13:54:20.467227 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:54:20.467134 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda45139c3_0352_430c_acdc_6fe24607c5fc.slice/crio-4c8758a629c4eb780efabcda764690820b1a0a1dfc3771ad093aba6f0771f052 WatchSource:0}: Error finding container 4c8758a629c4eb780efabcda764690820b1a0a1dfc3771ad093aba6f0771f052: Status 404 returned error can't find the container with id 4c8758a629c4eb780efabcda764690820b1a0a1dfc3771ad093aba6f0771f052 Jun 03 13:54:20.619678 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.619511 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:20.619784 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.619514 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:20.619861 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.619512 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:20.622438 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.622191 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 13:54:20.622438 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.622210 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n9kxv\"" Jun 03 13:54:20.622438 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.622216 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 13:54:20.622438 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.622256 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tj8sj\"" Jun 03 13:54:20.622438 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.622221 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 13:54:20.622836 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.622823 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 03 13:54:20.689998 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.689965 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:54:20.690141 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.690064 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:20.690141 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.690092 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:20.690141 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:20.690107 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 03 13:54:20.690235 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:20.690187 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert podName:4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b nodeName:}" failed. No retries permitted until 2026-06-03 13:54:22.690170036 +0000 UTC m=+38.634813978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-9qlnz" (UID: "4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b") : secret "networking-console-plugin-cert" not found Jun 03 13:54:20.690235 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:20.690200 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:54:20.690324 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:20.690242 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls podName:6724d1a1-68ea-4f1d-a782-7bbaf376213c nodeName:}" failed. No retries permitted until 2026-06-03 13:54:22.690228443 +0000 UTC m=+38.634872379 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls") pod "dns-default-7d2wt" (UID: "6724d1a1-68ea-4f1d-a782-7bbaf376213c") : secret "dns-default-metrics-tls" not found Jun 03 13:54:20.690324 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:20.690242 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:54:20.690324 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:20.690263 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cd7686cb6-7bhjc: secret "image-registry-tls" not found Jun 03 13:54:20.690324 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:20.690309 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls podName:063b7d6f-5926-4d42-a29c-3ed54eaeb18a nodeName:}" failed. No retries permitted until 2026-06-03 13:54:22.690293937 +0000 UTC m=+38.634937873 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls") pod "image-registry-7cd7686cb6-7bhjc" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a") : secret "image-registry-tls" not found Jun 03 13:54:20.761947 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.761915 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f5chv" event={"ID":"1d997ce2-6758-4937-8e63-dad6bf7b689a","Type":"ContainerStarted","Data":"1498a28d0782106161c21b0048b648989b5d1b9c83b4d7f94397213b1e5fd3da"} Jun 03 13:54:20.762966 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.762943 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" event={"ID":"611161c3-524f-4088-8e3f-67d41f5350da","Type":"ContainerStarted","Data":"176e6f71dae1a621fe6f1835ea9a4064b570086291f024fce2b54c6ae4ccc82c"} Jun 03 13:54:20.763950 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.763918 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" event={"ID":"a45139c3-0352-430c-acdc-6fe24607c5fc","Type":"ContainerStarted","Data":"4c8758a629c4eb780efabcda764690820b1a0a1dfc3771ad093aba6f0771f052"} Jun 03 13:54:20.764803 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.764786 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" event={"ID":"dad994ec-200d-446b-8078-99e71623f5e5","Type":"ContainerStarted","Data":"4b2d48a34c82f6c8658f8d14ca0483b973c656b3ede0bbe050f2edce7122e72e"} Jun 03 13:54:20.791451 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:20.791432 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:54:20.791558 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:20.791544 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:54:20.791616 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:20.791606 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert podName:04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea nodeName:}" failed. No retries permitted until 2026-06-03 13:54:22.791587255 +0000 UTC m=+38.736231211 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert") pod "ingress-canary-p66h6" (UID: "04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea") : secret "canary-serving-cert" not found Jun 03 13:54:21.770551 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:21.770445 2565 generic.go:358] "Generic (PLEG): container finished" podID="1d997ce2-6758-4937-8e63-dad6bf7b689a" containerID="1498a28d0782106161c21b0048b648989b5d1b9c83b4d7f94397213b1e5fd3da" exitCode=0 Jun 03 13:54:21.770551 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:21.770495 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f5chv" event={"ID":"1d997ce2-6758-4937-8e63-dad6bf7b689a","Type":"ContainerDied","Data":"1498a28d0782106161c21b0048b648989b5d1b9c83b4d7f94397213b1e5fd3da"} Jun 03 13:54:22.710133 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:22.710087 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:22.710506 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:22.710141 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:22.710506 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:22.710204 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:54:22.710506 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:22.710374 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 03 13:54:22.710506 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:22.710442 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert podName:4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b nodeName:}" failed. No retries permitted until 2026-06-03 13:54:26.710421823 +0000 UTC m=+42.655065777 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-9qlnz" (UID: "4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b") : secret "networking-console-plugin-cert" not found Jun 03 13:54:22.711024 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:22.710863 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:54:22.711024 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:22.710885 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cd7686cb6-7bhjc: secret "image-registry-tls" not found Jun 03 13:54:22.711024 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:22.710930 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls podName:063b7d6f-5926-4d42-a29c-3ed54eaeb18a nodeName:}" failed. No retries permitted until 2026-06-03 13:54:26.710915072 +0000 UTC m=+42.655559007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls") pod "image-registry-7cd7686cb6-7bhjc" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a") : secret "image-registry-tls" not found Jun 03 13:54:22.711024 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:22.710861 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:54:22.711024 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:22.710968 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls podName:6724d1a1-68ea-4f1d-a782-7bbaf376213c nodeName:}" failed. No retries permitted until 2026-06-03 13:54:26.71095865 +0000 UTC m=+42.655602586 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls") pod "dns-default-7d2wt" (UID: "6724d1a1-68ea-4f1d-a782-7bbaf376213c") : secret "dns-default-metrics-tls" not found Jun 03 13:54:22.776613 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:22.776541 2565 generic.go:358] "Generic (PLEG): container finished" podID="1d997ce2-6758-4937-8e63-dad6bf7b689a" containerID="43a42a33813758a8d999d47e36e1f4c0375804b9778f07a748468a07ac61b641" exitCode=0 Jun 03 13:54:22.777015 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:22.776611 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f5chv" event={"ID":"1d997ce2-6758-4937-8e63-dad6bf7b689a","Type":"ContainerDied","Data":"43a42a33813758a8d999d47e36e1f4c0375804b9778f07a748468a07ac61b641"} Jun 03 13:54:22.811509 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:22.811471 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:54:22.811629 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:22.811618 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:54:22.811685 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:22.811672 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert podName:04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea nodeName:}" failed. No retries permitted until 2026-06-03 13:54:26.811655181 +0000 UTC m=+42.756299120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert") pod "ingress-canary-p66h6" (UID: "04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea") : secret "canary-serving-cert" not found Jun 03 13:54:26.742580 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:26.742536 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:26.742580 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:26.742585 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:26.743063 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:26.742636 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:54:26.743063 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:26.742699 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:54:26.743063 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:26.742724 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cd7686cb6-7bhjc: secret "image-registry-tls" not found Jun 03 13:54:26.743063 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:26.742761 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:54:26.743063 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:26.742784 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 03 13:54:26.743063 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:26.742806 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls podName:063b7d6f-5926-4d42-a29c-3ed54eaeb18a nodeName:}" failed. No retries permitted until 2026-06-03 13:54:34.742784244 +0000 UTC m=+50.687428185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls") pod "image-registry-7cd7686cb6-7bhjc" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a") : secret "image-registry-tls" not found Jun 03 13:54:26.743063 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:26.742826 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert podName:4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b nodeName:}" failed. No retries permitted until 2026-06-03 13:54:34.742813797 +0000 UTC m=+50.687457730 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-9qlnz" (UID: "4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b") : secret "networking-console-plugin-cert" not found Jun 03 13:54:26.743063 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:26.742843 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls podName:6724d1a1-68ea-4f1d-a782-7bbaf376213c nodeName:}" failed. No retries permitted until 2026-06-03 13:54:34.742833245 +0000 UTC m=+50.687477184 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls") pod "dns-default-7d2wt" (UID: "6724d1a1-68ea-4f1d-a782-7bbaf376213c") : secret "dns-default-metrics-tls" not found Jun 03 13:54:26.843451 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:26.843416 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:54:26.843611 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:26.843559 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:54:26.843679 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:26.843615 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert podName:04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea nodeName:}" failed. No retries permitted until 2026-06-03 13:54:34.843597318 +0000 UTC m=+50.788241252 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert") pod "ingress-canary-p66h6" (UID: "04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea") : secret "canary-serving-cert" not found Jun 03 13:54:28.790022 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:28.789772 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f5chv" event={"ID":"1d997ce2-6758-4937-8e63-dad6bf7b689a","Type":"ContainerStarted","Data":"acc7eb2c69595a0fadea8fdd610aa60176ab75ad4f1acb837ce6b7b9f2861ae4"} Jun 03 13:54:28.791138 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:28.791113 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" event={"ID":"611161c3-524f-4088-8e3f-67d41f5350da","Type":"ContainerStarted","Data":"62fe90700041a2caf51ff3dec6019c26e6961da20206b93e5d4249e543bd3f37"} Jun 03 13:54:28.791355 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:28.791323 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:54:28.792475 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:28.792452 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" event={"ID":"a45139c3-0352-430c-acdc-6fe24607c5fc","Type":"ContainerStarted","Data":"773c09d80bc49bd317ea0ad0ae2d69596bb160df849af05ffc0460738bb45cb1"} Jun 03 13:54:28.793230 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:28.793211 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:54:28.793805 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:28.793785 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" event={"ID":"dad994ec-200d-446b-8078-99e71623f5e5","Type":"ContainerStarted","Data":"a53ac9135833bfc0429936c7baec50cce4331b9f8391d360df34721f2374b0bc"} Jun 03 13:54:28.853396 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:28.853334 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-f5chv" podStartSLOduration=10.938180155 podStartE2EDuration="43.853322349s" podCreationTimestamp="2026-06-03 13:53:45 +0000 UTC" firstStartedPulling="2026-06-03 13:53:47.580960641 +0000 UTC m=+3.525604577" lastFinishedPulling="2026-06-03 13:54:20.496102835 +0000 UTC m=+36.440746771" observedRunningTime="2026-06-03 13:54:28.8531263 +0000 UTC m=+44.797770255" watchObservedRunningTime="2026-06-03 13:54:28.853322349 +0000 UTC m=+44.797966283" Jun 03 13:54:28.876697 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:28.876659 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" podStartSLOduration=8.090978492 podStartE2EDuration="15.876644502s" podCreationTimestamp="2026-06-03 13:54:13 +0000 UTC" firstStartedPulling="2026-06-03 13:54:20.472820383 +0000 UTC m=+36.417464321" lastFinishedPulling="2026-06-03 13:54:28.258486393 +0000 UTC m=+44.203130331" observedRunningTime="2026-06-03 13:54:28.876225887 +0000 UTC m=+44.820869841" watchObservedRunningTime="2026-06-03 13:54:28.876644502 +0000 UTC m=+44.821288457" Jun 03 13:54:28.897989 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:28.897955 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" podStartSLOduration=8.121776372 podStartE2EDuration="15.897945883s" podCreationTimestamp="2026-06-03 13:54:13 +0000 UTC" firstStartedPulling="2026-06-03 13:54:20.472747612 +0000 UTC m=+36.417391548" lastFinishedPulling="2026-06-03 13:54:28.248917111 +0000 UTC m=+44.193561059" observedRunningTime="2026-06-03 13:54:28.897541149 +0000 UTC m=+44.842185104" watchObservedRunningTime="2026-06-03 13:54:28.897945883 +0000 UTC m=+44.842589838" Jun 03 13:54:31.481415 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:31.481327 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:31.485515 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:31.485492 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/561a2ef6-f963-4946-bdf9-8336eaae20d6-original-pull-secret\") pod \"global-pull-secret-syncer-nsnhl\" (UID: \"561a2ef6-f963-4946-bdf9-8336eaae20d6\") " pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:31.749449 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:31.749370 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nsnhl" Jun 03 13:54:31.802703 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:31.802671 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" event={"ID":"a45139c3-0352-430c-acdc-6fe24607c5fc","Type":"ContainerStarted","Data":"35f94c785fb9183f45dc8755f29fd8a45802bdba26bdf1945603ce14cc93e8ed"} Jun 03 13:54:31.802827 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:31.802713 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" event={"ID":"a45139c3-0352-430c-acdc-6fe24607c5fc","Type":"ContainerStarted","Data":"5d6715de196cf18993ae903620f7f1c58db127554f8e18fc8b142e636981a9e3"} Jun 03 13:54:31.822407 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:31.822215 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" podStartSLOduration=8.065845797 podStartE2EDuration="18.822202014s" podCreationTimestamp="2026-06-03 13:54:13 +0000 UTC" firstStartedPulling="2026-06-03 13:54:20.472689596 +0000 UTC m=+36.417333541" lastFinishedPulling="2026-06-03 13:54:31.229045811 +0000 UTC m=+47.173689758" observedRunningTime="2026-06-03 13:54:31.822085368 +0000 UTC m=+47.766729324" watchObservedRunningTime="2026-06-03 13:54:31.822202014 +0000 UTC m=+47.766845966" Jun 03 13:54:31.865852 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:31.865823 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nsnhl"] Jun 03 13:54:31.869365 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:54:31.869324 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod561a2ef6_f963_4946_bdf9_8336eaae20d6.slice/crio-38955bc3811fc83bce98bc8fd32a4c995b1cba02d91c665e32cafe2f84f98f28 WatchSource:0}: Error finding container 38955bc3811fc83bce98bc8fd32a4c995b1cba02d91c665e32cafe2f84f98f28: Status 404 returned error can't find the container with id 38955bc3811fc83bce98bc8fd32a4c995b1cba02d91c665e32cafe2f84f98f28 Jun 03 13:54:32.806352 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:32.806301 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nsnhl" event={"ID":"561a2ef6-f963-4946-bdf9-8336eaae20d6","Type":"ContainerStarted","Data":"38955bc3811fc83bce98bc8fd32a4c995b1cba02d91c665e32cafe2f84f98f28"} Jun 03 13:54:34.809243 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:34.809195 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:54:34.809860 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:34.809318 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:34.809860 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:34.809368 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:34.809860 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:34.809374 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 03 13:54:34.809860 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:34.809449 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:54:34.809860 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:34.809464 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cd7686cb6-7bhjc: secret "image-registry-tls" not found Jun 03 13:54:34.809860 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:34.809498 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:54:34.809860 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:34.809451 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert podName:4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b nodeName:}" failed. No retries permitted until 2026-06-03 13:54:50.8094309 +0000 UTC m=+66.754074836 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-9qlnz" (UID: "4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b") : secret "networking-console-plugin-cert" not found Jun 03 13:54:34.809860 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:34.809533 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls podName:063b7d6f-5926-4d42-a29c-3ed54eaeb18a nodeName:}" failed. No retries permitted until 2026-06-03 13:54:50.809520883 +0000 UTC m=+66.754164827 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls") pod "image-registry-7cd7686cb6-7bhjc" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a") : secret "image-registry-tls" not found Jun 03 13:54:34.809860 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:34.809562 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls podName:6724d1a1-68ea-4f1d-a782-7bbaf376213c nodeName:}" failed. No retries permitted until 2026-06-03 13:54:50.809545877 +0000 UTC m=+66.754189811 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls") pod "dns-default-7d2wt" (UID: "6724d1a1-68ea-4f1d-a782-7bbaf376213c") : secret "dns-default-metrics-tls" not found Jun 03 13:54:34.910293 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:34.910258 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:54:34.910466 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:34.910434 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:54:34.910529 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:34.910507 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert podName:04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea nodeName:}" failed. No retries permitted until 2026-06-03 13:54:50.910489875 +0000 UTC m=+66.855133808 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert") pod "ingress-canary-p66h6" (UID: "04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea") : secret "canary-serving-cert" not found Jun 03 13:54:36.817028 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:36.816991 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nsnhl" event={"ID":"561a2ef6-f963-4946-bdf9-8336eaae20d6","Type":"ContainerStarted","Data":"435d81dc0e1ddbab9df7cd69315aba1b1d9bf3240a3499bc38273efb1c39808b"} Jun 03 13:54:44.757853 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:44.757676 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gzz7s" Jun 03 13:54:44.785879 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:44.785827 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nsnhl" podStartSLOduration=41.431723473 podStartE2EDuration="45.785813953s" podCreationTimestamp="2026-06-03 13:53:59 +0000 UTC" firstStartedPulling="2026-06-03 13:54:31.871034609 +0000 UTC m=+47.815678547" lastFinishedPulling="2026-06-03 13:54:36.225125095 +0000 UTC m=+52.169769027" observedRunningTime="2026-06-03 13:54:36.832033704 +0000 UTC m=+52.776677660" watchObservedRunningTime="2026-06-03 13:54:44.785813953 +0000 UTC m=+60.730457901" Jun 03 13:54:50.327024 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.326985 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:54:50.329650 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.329631 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 13:54:50.337686 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:50.337671 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 03 13:54:50.337733 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:50.337722 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs podName:277e2d8b-f5ab-4f08-bb7e-b6981b3932fc nodeName:}" failed. No retries permitted until 2026-06-03 13:55:54.337707473 +0000 UTC m=+130.282351406 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs") pod "network-metrics-daemon-5m5l8" (UID: "277e2d8b-f5ab-4f08-bb7e-b6981b3932fc") : secret "metrics-daemon-secret" not found Jun 03 13:54:50.427495 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.427470 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h84w6\" (UniqueName: \"kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6\") pod \"network-check-target-qsp8b\" (UID: \"8eb13e0d-051c-4a0e-a19d-6252080682a9\") " pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:50.430203 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.430187 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 13:54:50.440746 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.440730 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 13:54:50.452232 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.452205 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h84w6\" (UniqueName: \"kubernetes.io/projected/8eb13e0d-051c-4a0e-a19d-6252080682a9-kube-api-access-h84w6\") pod \"network-check-target-qsp8b\" (UID: \"8eb13e0d-051c-4a0e-a19d-6252080682a9\") " pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:50.636150 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.636096 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n9kxv\"" Jun 03 13:54:50.644367 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.644335 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:54:50.759525 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.759495 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qsp8b"] Jun 03 13:54:50.762773 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:54:50.762747 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eb13e0d_051c_4a0e_a19d_6252080682a9.slice/crio-e3054adc07144783d6110c2d6989d5eb92688e5381923782f8479d045118292d WatchSource:0}: Error finding container e3054adc07144783d6110c2d6989d5eb92688e5381923782f8479d045118292d: Status 404 returned error can't find the container with id e3054adc07144783d6110c2d6989d5eb92688e5381923782f8479d045118292d Jun 03 13:54:50.829683 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.829654 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:54:50.829775 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.829721 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:54:50.829775 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.829746 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:54:50.829860 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:50.829807 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 03 13:54:50.829860 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:50.829842 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:54:50.829860 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:50.829857 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:54:50.829949 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:50.829869 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cd7686cb6-7bhjc: secret "image-registry-tls" not found Jun 03 13:54:50.829949 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:50.829869 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert podName:4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b nodeName:}" failed. No retries permitted until 2026-06-03 13:55:22.829852049 +0000 UTC m=+98.774495981 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-9qlnz" (UID: "4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b") : secret "networking-console-plugin-cert" not found Jun 03 13:54:50.829949 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:50.829924 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls podName:6724d1a1-68ea-4f1d-a782-7bbaf376213c nodeName:}" failed. No retries permitted until 2026-06-03 13:55:22.829912309 +0000 UTC m=+98.774556245 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls") pod "dns-default-7d2wt" (UID: "6724d1a1-68ea-4f1d-a782-7bbaf376213c") : secret "dns-default-metrics-tls" not found Jun 03 13:54:50.829949 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:50.829938 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls podName:063b7d6f-5926-4d42-a29c-3ed54eaeb18a nodeName:}" failed. No retries permitted until 2026-06-03 13:55:22.82992849 +0000 UTC m=+98.774572426 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls") pod "image-registry-7cd7686cb6-7bhjc" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a") : secret "image-registry-tls" not found Jun 03 13:54:50.855194 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.855167 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qsp8b" event={"ID":"8eb13e0d-051c-4a0e-a19d-6252080682a9","Type":"ContainerStarted","Data":"e3054adc07144783d6110c2d6989d5eb92688e5381923782f8479d045118292d"} Jun 03 13:54:50.930893 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:50.930841 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:54:50.930987 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:50.930973 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:54:50.931027 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:54:50.931023 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert podName:04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea nodeName:}" failed. No retries permitted until 2026-06-03 13:55:22.93100728 +0000 UTC m=+98.875651223 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert") pod "ingress-canary-p66h6" (UID: "04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea") : secret "canary-serving-cert" not found Jun 03 13:54:54.866293 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:54.866256 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qsp8b" event={"ID":"8eb13e0d-051c-4a0e-a19d-6252080682a9","Type":"ContainerStarted","Data":"f1b0ba7357ca76684e39ae94fc46638284372ed38e47959382d5d34eeb84439c"} Jun 03 13:54:54.866747 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:54:54.866413 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:55:22.869645 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:55:22.869549 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:55:22.869645 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:55:22.869598 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:55:22.869645 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:55:22.869631 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:55:22.870085 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:55:22.869709 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:55:22.870085 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:55:22.869735 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cd7686cb6-7bhjc: secret "image-registry-tls" not found Jun 03 13:55:22.870085 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:55:22.869739 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 03 13:55:22.870085 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:55:22.869749 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:55:22.870085 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:55:22.869809 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls podName:063b7d6f-5926-4d42-a29c-3ed54eaeb18a nodeName:}" failed. No retries permitted until 2026-06-03 13:56:26.869786306 +0000 UTC m=+162.814430243 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls") pod "image-registry-7cd7686cb6-7bhjc" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a") : secret "image-registry-tls" not found Jun 03 13:55:22.870085 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:55:22.869825 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert podName:4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b nodeName:}" failed. No retries permitted until 2026-06-03 13:56:26.869818026 +0000 UTC m=+162.814461959 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-9qlnz" (UID: "4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b") : secret "networking-console-plugin-cert" not found Jun 03 13:55:22.870085 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:55:22.869835 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls podName:6724d1a1-68ea-4f1d-a782-7bbaf376213c nodeName:}" failed. No retries permitted until 2026-06-03 13:56:26.869829504 +0000 UTC m=+162.814473437 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls") pod "dns-default-7d2wt" (UID: "6724d1a1-68ea-4f1d-a782-7bbaf376213c") : secret "dns-default-metrics-tls" not found Jun 03 13:55:22.970861 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:55:22.970829 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:55:22.971048 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:55:22.970986 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:55:22.971116 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:55:22.971053 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert podName:04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea nodeName:}" failed. No retries permitted until 2026-06-03 13:56:26.971033888 +0000 UTC m=+162.915677834 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert") pod "ingress-canary-p66h6" (UID: "04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea") : secret "canary-serving-cert" not found Jun 03 13:55:25.871453 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:55:25.871422 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qsp8b" Jun 03 13:55:25.887612 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:55:25.887566 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qsp8b" podStartSLOduration=98.329179071 podStartE2EDuration="1m41.88755371s" podCreationTimestamp="2026-06-03 13:53:44 +0000 UTC" firstStartedPulling="2026-06-03 13:54:50.76525451 +0000 UTC m=+66.709898444" lastFinishedPulling="2026-06-03 13:54:54.32362915 +0000 UTC m=+70.268273083" observedRunningTime="2026-06-03 13:54:54.88262139 +0000 UTC m=+70.827265345" watchObservedRunningTime="2026-06-03 13:55:25.88755371 +0000 UTC m=+101.832197665" Jun 03 13:55:54.413811 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:55:54.413773 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:55:54.414277 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:55:54.413910 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 03 13:55:54.414277 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:55:54.413975 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs podName:277e2d8b-f5ab-4f08-bb7e-b6981b3932fc nodeName:}" failed. No retries permitted until 2026-06-03 13:57:56.413960108 +0000 UTC m=+252.358604046 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs") pod "network-metrics-daemon-5m5l8" (UID: "277e2d8b-f5ab-4f08-bb7e-b6981b3932fc") : secret "metrics-daemon-secret" not found Jun 03 13:56:12.024437 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:12.024406 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gbtxm_8c5d4af8-6b15-491f-9c16-884f366947e3/dns-node-resolver/0.log" Jun 03 13:56:12.824716 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:12.824683 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6ns2r_dedc278e-adec-4d58-bafb-047e9a36c84c/node-ca/0.log" Jun 03 13:56:21.950829 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:21.950784 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" podUID="4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b" Jun 03 13:56:21.962956 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:21.962922 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" podUID="063b7d6f-5926-4d42-a29c-3ed54eaeb18a" Jun 03 13:56:22.001434 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:22.001387 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7d2wt" podUID="6724d1a1-68ea-4f1d-a782-7bbaf376213c" Jun 03 13:56:22.062922 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:22.062894 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:56:22.063100 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:22.062894 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:56:22.063100 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:22.062894 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7d2wt" Jun 03 13:56:22.083521 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:22.083497 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-p66h6" podUID="04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea" Jun 03 13:56:23.065461 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:23.065432 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:56:23.644110 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:23.644064 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5m5l8" podUID="277e2d8b-f5ab-4f08-bb7e-b6981b3932fc" Jun 03 13:56:26.959276 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:26.959237 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls\") pod \"image-registry-7cd7686cb6-7bhjc\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:56:26.959276 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:26.959282 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:56:26.959740 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:26.959333 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:56:26.959740 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:26.959416 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 13:56:26.959740 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:26.959432 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cd7686cb6-7bhjc: secret "image-registry-tls" not found Jun 03 13:56:26.959740 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:26.959455 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 13:56:26.959740 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:26.959487 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 03 13:56:26.959740 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:26.959497 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls podName:063b7d6f-5926-4d42-a29c-3ed54eaeb18a nodeName:}" failed. No retries permitted until 2026-06-03 13:58:28.959478249 +0000 UTC m=+284.904122190 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls") pod "image-registry-7cd7686cb6-7bhjc" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a") : secret "image-registry-tls" not found Jun 03 13:56:26.959740 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:26.959516 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls podName:6724d1a1-68ea-4f1d-a782-7bbaf376213c nodeName:}" failed. No retries permitted until 2026-06-03 13:58:28.959505515 +0000 UTC m=+284.904149451 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls") pod "dns-default-7d2wt" (UID: "6724d1a1-68ea-4f1d-a782-7bbaf376213c") : secret "dns-default-metrics-tls" not found Jun 03 13:56:26.959740 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:26.959533 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert podName:4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b nodeName:}" failed. No retries permitted until 2026-06-03 13:58:28.959522727 +0000 UTC m=+284.904166664 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-9qlnz" (UID: "4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b") : secret "networking-console-plugin-cert" not found Jun 03 13:56:27.060143 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:27.060108 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:56:27.060330 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:27.060249 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 13:56:27.060330 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:56:27.060304 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert podName:04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea nodeName:}" failed. No retries permitted until 2026-06-03 13:58:29.060288791 +0000 UTC m=+285.004932730 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert") pod "ingress-canary-p66h6" (UID: "04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea") : secret "canary-serving-cert" not found Jun 03 13:56:28.791741 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:28.791696 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" podUID="611161c3-524f-4088-8e3f-67d41f5350da" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/readyz\": dial tcp 10.132.0.9:8000: connect: connection refused" Jun 03 13:56:29.080084 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:29.080000 2565 generic.go:358] "Generic (PLEG): container finished" podID="611161c3-524f-4088-8e3f-67d41f5350da" containerID="62fe90700041a2caf51ff3dec6019c26e6961da20206b93e5d4249e543bd3f37" exitCode=1 Jun 03 13:56:29.080084 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:29.080058 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" event={"ID":"611161c3-524f-4088-8e3f-67d41f5350da","Type":"ContainerDied","Data":"62fe90700041a2caf51ff3dec6019c26e6961da20206b93e5d4249e543bd3f37"} Jun 03 13:56:29.080447 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:29.080423 2565 scope.go:117] "RemoveContainer" containerID="62fe90700041a2caf51ff3dec6019c26e6961da20206b93e5d4249e543bd3f37" Jun 03 13:56:29.081252 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:29.081232 2565 generic.go:358] "Generic (PLEG): container finished" podID="dad994ec-200d-446b-8078-99e71623f5e5" containerID="a53ac9135833bfc0429936c7baec50cce4331b9f8391d360df34721f2374b0bc" exitCode=255 Jun 03 13:56:29.081321 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:29.081290 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" event={"ID":"dad994ec-200d-446b-8078-99e71623f5e5","Type":"ContainerDied","Data":"a53ac9135833bfc0429936c7baec50cce4331b9f8391d360df34721f2374b0bc"} Jun 03 13:56:29.081584 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:29.081568 2565 scope.go:117] "RemoveContainer" containerID="a53ac9135833bfc0429936c7baec50cce4331b9f8391d360df34721f2374b0bc" Jun 03 13:56:29.287270 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:29.287237 2565 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:56:29.296416 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:29.296392 2565 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" Jun 03 13:56:30.088381 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:30.088324 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" event={"ID":"611161c3-524f-4088-8e3f-67d41f5350da","Type":"ContainerStarted","Data":"bd4f254c4d38e2fd0ab06437c36bf3b03199795dadc74e13bf450704506c620d"} Jun 03 13:56:30.088826 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:30.088524 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:56:30.089471 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:30.089451 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44ffdf6-gj2q4" Jun 03 13:56:30.090044 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:30.090025 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b4d5f6786-dkr5l" event={"ID":"dad994ec-200d-446b-8078-99e71623f5e5","Type":"ContainerStarted","Data":"9b88dcc2784f02f0feee5d3d072c658f9ec184c65fba1b36c7fe3b8d7edd5f91"} Jun 03 13:56:35.619007 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:35.618929 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:56:44.073745 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.073710 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4jdkm"] Jun 03 13:56:44.076879 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.076857 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.079248 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.079228 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 03 13:56:44.079549 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.079532 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 03 13:56:44.080092 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.080074 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 03 13:56:44.080162 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.080134 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8c6lj\"" Jun 03 13:56:44.081084 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.081064 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 03 13:56:44.092594 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.092573 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4jdkm"] Jun 03 13:56:44.194084 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.194055 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49e68b9c-5f36-4ff4-a96e-550613b82dc3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.194231 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.194111 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l9lg\" (UniqueName: \"kubernetes.io/projected/49e68b9c-5f36-4ff4-a96e-550613b82dc3-kube-api-access-4l9lg\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.194231 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.194155 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49e68b9c-5f36-4ff4-a96e-550613b82dc3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.194231 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.194208 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/49e68b9c-5f36-4ff4-a96e-550613b82dc3-crio-socket\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.194381 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.194234 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/49e68b9c-5f36-4ff4-a96e-550613b82dc3-data-volume\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.295095 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.295065 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4l9lg\" (UniqueName: \"kubernetes.io/projected/49e68b9c-5f36-4ff4-a96e-550613b82dc3-kube-api-access-4l9lg\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.295229 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.295106 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49e68b9c-5f36-4ff4-a96e-550613b82dc3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.295229 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.295141 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/49e68b9c-5f36-4ff4-a96e-550613b82dc3-crio-socket\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.295229 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.295161 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/49e68b9c-5f36-4ff4-a96e-550613b82dc3-data-volume\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.295229 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.295196 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49e68b9c-5f36-4ff4-a96e-550613b82dc3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.295774 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.295746 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/49e68b9c-5f36-4ff4-a96e-550613b82dc3-crio-socket\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.299051 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.295999 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/49e68b9c-5f36-4ff4-a96e-550613b82dc3-data-volume\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.299051 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.296174 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49e68b9c-5f36-4ff4-a96e-550613b82dc3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.300836 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.300815 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49e68b9c-5f36-4ff4-a96e-550613b82dc3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.305736 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.305711 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l9lg\" (UniqueName: \"kubernetes.io/projected/49e68b9c-5f36-4ff4-a96e-550613b82dc3-kube-api-access-4l9lg\") pod \"insights-runtime-extractor-4jdkm\" (UID: \"49e68b9c-5f36-4ff4-a96e-550613b82dc3\") " pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.385356 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.385282 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4jdkm" Jun 03 13:56:44.517106 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:44.517076 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4jdkm"] Jun 03 13:56:44.520399 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:56:44.520373 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e68b9c_5f36_4ff4_a96e_550613b82dc3.slice/crio-852e41660db2b2df2811f85f4ccce9ab13572f94c02c1fecc624fcd8a1ea063f WatchSource:0}: Error finding container 852e41660db2b2df2811f85f4ccce9ab13572f94c02c1fecc624fcd8a1ea063f: Status 404 returned error can't find the container with id 852e41660db2b2df2811f85f4ccce9ab13572f94c02c1fecc624fcd8a1ea063f Jun 03 13:56:45.125682 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:45.125641 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4jdkm" event={"ID":"49e68b9c-5f36-4ff4-a96e-550613b82dc3","Type":"ContainerStarted","Data":"d70618353eceba6369ec6118d9b3089819889672882ed3bae70d7fea920af4aa"} Jun 03 13:56:45.125682 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:45.125687 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4jdkm" event={"ID":"49e68b9c-5f36-4ff4-a96e-550613b82dc3","Type":"ContainerStarted","Data":"852e41660db2b2df2811f85f4ccce9ab13572f94c02c1fecc624fcd8a1ea063f"} Jun 03 13:56:46.129547 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:46.129513 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4jdkm" event={"ID":"49e68b9c-5f36-4ff4-a96e-550613b82dc3","Type":"ContainerStarted","Data":"f7f90b275a90a03d0a0159e843e9125518f8d5bfa8f60da0366f307a3e32df9a"} Jun 03 13:56:47.133640 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:47.133551 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4jdkm" event={"ID":"49e68b9c-5f36-4ff4-a96e-550613b82dc3","Type":"ContainerStarted","Data":"60915cbe3d2b08c72a09459f0dd7c4ce6d894052366ef41c9dc8982fd4348f1b"} Jun 03 13:56:47.164076 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:47.164004 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4jdkm" podStartSLOduration=0.947743352 podStartE2EDuration="3.163989436s" podCreationTimestamp="2026-06-03 13:56:44 +0000 UTC" firstStartedPulling="2026-06-03 13:56:44.573579904 +0000 UTC m=+180.518223837" lastFinishedPulling="2026-06-03 13:56:46.789825988 +0000 UTC m=+182.734469921" observedRunningTime="2026-06-03 13:56:47.163723333 +0000 UTC m=+183.108367288" watchObservedRunningTime="2026-06-03 13:56:47.163989436 +0000 UTC m=+183.108633390" Jun 03 13:56:51.681856 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.681823 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5wd67"] Jun 03 13:56:51.685217 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.685200 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.687829 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.687805 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 03 13:56:51.687829 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.687816 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 03 13:56:51.687829 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.687820 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 03 13:56:51.688031 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.687873 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 03 13:56:51.688616 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.688595 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vm544\"" Jun 03 13:56:51.688691 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.688597 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 03 13:56:51.688691 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.688684 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 03 13:56:51.857629 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.857597 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-accelerators-collector-config\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.857788 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.857642 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-textfile\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.857788 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.857660 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f30986ba-b02c-4645-be24-c5867f3a7f03-root\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.857788 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.857695 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-tls\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.857788 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.857711 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.857788 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.857731 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jncs4\" (UniqueName: \"kubernetes.io/projected/f30986ba-b02c-4645-be24-c5867f3a7f03-kube-api-access-jncs4\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.857944 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.857796 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-wtmp\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.857944 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.857847 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f30986ba-b02c-4645-be24-c5867f3a7f03-metrics-client-ca\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.857944 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.857887 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f30986ba-b02c-4645-be24-c5867f3a7f03-sys\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.958506 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.958472 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-textfile\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.958686 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.958514 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f30986ba-b02c-4645-be24-c5867f3a7f03-root\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.958686 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.958556 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-tls\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.958686 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.958584 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.958686 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.958648 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f30986ba-b02c-4645-be24-c5867f3a7f03-root\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.958893 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.958697 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jncs4\" (UniqueName: \"kubernetes.io/projected/f30986ba-b02c-4645-be24-c5867f3a7f03-kube-api-access-jncs4\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.958893 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.958727 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-wtmp\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.958893 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.958766 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f30986ba-b02c-4645-be24-c5867f3a7f03-metrics-client-ca\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.958893 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.958811 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f30986ba-b02c-4645-be24-c5867f3a7f03-sys\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.958893 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.958818 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-textfile\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.959128 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.958897 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f30986ba-b02c-4645-be24-c5867f3a7f03-sys\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.959128 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.958896 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-accelerators-collector-config\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.959128 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.958930 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-wtmp\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.959394 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.959377 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f30986ba-b02c-4645-be24-c5867f3a7f03-metrics-client-ca\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.959457 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.959414 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-accelerators-collector-config\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.961424 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.961400 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.961604 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.961583 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f30986ba-b02c-4645-be24-c5867f3a7f03-node-exporter-tls\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.966757 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.966724 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jncs4\" (UniqueName: \"kubernetes.io/projected/f30986ba-b02c-4645-be24-c5867f3a7f03-kube-api-access-jncs4\") pod \"node-exporter-5wd67\" (UID: \"f30986ba-b02c-4645-be24-c5867f3a7f03\") " pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:51.993784 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:51.993762 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5wd67" Jun 03 13:56:52.001187 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:56:52.001163 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf30986ba_b02c_4645_be24_c5867f3a7f03.slice/crio-171b3784261ac0d253186c48f4da442dbd005e607ea2a9a56445546a0e69235d WatchSource:0}: Error finding container 171b3784261ac0d253186c48f4da442dbd005e607ea2a9a56445546a0e69235d: Status 404 returned error can't find the container with id 171b3784261ac0d253186c48f4da442dbd005e607ea2a9a56445546a0e69235d Jun 03 13:56:52.146318 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:52.146279 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wd67" event={"ID":"f30986ba-b02c-4645-be24-c5867f3a7f03","Type":"ContainerStarted","Data":"171b3784261ac0d253186c48f4da442dbd005e607ea2a9a56445546a0e69235d"} Jun 03 13:56:54.152473 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:54.152440 2565 generic.go:358] "Generic (PLEG): container finished" podID="f30986ba-b02c-4645-be24-c5867f3a7f03" containerID="aeba4d6b54623bed2a5fc2da8be90d8588f893bba11bdb1a20f5bd2bc6cc9341" exitCode=0 Jun 03 13:56:54.152839 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:54.152516 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wd67" event={"ID":"f30986ba-b02c-4645-be24-c5867f3a7f03","Type":"ContainerDied","Data":"aeba4d6b54623bed2a5fc2da8be90d8588f893bba11bdb1a20f5bd2bc6cc9341"} Jun 03 13:56:55.156591 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:55.156549 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wd67" event={"ID":"f30986ba-b02c-4645-be24-c5867f3a7f03","Type":"ContainerStarted","Data":"aad67ca8a8b2a16192ed84062330807bf30aea4b103174a87d610cc18793a0c0"} Jun 03 13:56:55.156943 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:55.156597 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5wd67" event={"ID":"f30986ba-b02c-4645-be24-c5867f3a7f03","Type":"ContainerStarted","Data":"6d30c1bc0db6f2364f25f395faf386c5bf9bd6de6a1860fe3aa9c5b574f2aa31"} Jun 03 13:56:55.177703 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:55.177658 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5wd67" podStartSLOduration=3.054890283 podStartE2EDuration="4.177646262s" podCreationTimestamp="2026-06-03 13:56:51 +0000 UTC" firstStartedPulling="2026-06-03 13:56:52.002859863 +0000 UTC m=+187.947503799" lastFinishedPulling="2026-06-03 13:56:53.125615836 +0000 UTC m=+189.070259778" observedRunningTime="2026-06-03 13:56:55.176575966 +0000 UTC m=+191.121219924" watchObservedRunningTime="2026-06-03 13:56:55.177646262 +0000 UTC m=+191.122290216" Jun 03 13:56:59.279116 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:56:59.279072 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" podUID="a45139c3-0352-430c-acdc-6fe24607c5fc" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 03 13:57:05.989999 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:05.989959 2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cd7686cb6-7bhjc"] Jun 03 13:57:05.990431 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:57:05.990162 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" podUID="063b7d6f-5926-4d42-a29c-3ed54eaeb18a" Jun 03 13:57:06.182814 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.182785 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:57:06.190228 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.190204 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:57:06.277176 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.277087 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z529n\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-kube-api-access-z529n\") pod \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " Jun 03 13:57:06.277176 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.277125 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-bound-sa-token\") pod \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " Jun 03 13:57:06.277176 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.277160 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-installation-pull-secrets\") pod \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " Jun 03 13:57:06.277479 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.277196 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-trusted-ca\") pod \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " Jun 03 13:57:06.277479 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.277224 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-image-registry-private-configuration\") pod \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " Jun 03 13:57:06.277479 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.277260 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-ca-trust-extracted\") pod \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " Jun 03 13:57:06.277479 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.277300 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-certificates\") pod \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\" (UID: \"063b7d6f-5926-4d42-a29c-3ed54eaeb18a\") " Jun 03 13:57:06.277675 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.277608 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "063b7d6f-5926-4d42-a29c-3ed54eaeb18a" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 13:57:06.277830 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.277803 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "063b7d6f-5926-4d42-a29c-3ed54eaeb18a" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 13:57:06.277888 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.277823 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "063b7d6f-5926-4d42-a29c-3ed54eaeb18a" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 13:57:06.279711 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.279677 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "063b7d6f-5926-4d42-a29c-3ed54eaeb18a" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 13:57:06.279840 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.279754 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-kube-api-access-z529n" (OuterVolumeSpecName: "kube-api-access-z529n") pod "063b7d6f-5926-4d42-a29c-3ed54eaeb18a" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a"). InnerVolumeSpecName "kube-api-access-z529n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:57:06.279840 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.279826 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "063b7d6f-5926-4d42-a29c-3ed54eaeb18a" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 13:57:06.279936 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.279919 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "063b7d6f-5926-4d42-a29c-3ed54eaeb18a" (UID: "063b7d6f-5926-4d42-a29c-3ed54eaeb18a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 13:57:06.378497 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.378458 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z529n\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-kube-api-access-z529n\") on node \"ip-10-0-134-89.ec2.internal\" DevicePath \"\"" Jun 03 13:57:06.378497 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.378491 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-bound-sa-token\") on node \"ip-10-0-134-89.ec2.internal\" DevicePath \"\"" Jun 03 13:57:06.378497 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.378505 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-installation-pull-secrets\") on node \"ip-10-0-134-89.ec2.internal\" DevicePath \"\"" Jun 03 13:57:06.378731 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.378520 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-trusted-ca\") on node \"ip-10-0-134-89.ec2.internal\" DevicePath \"\"" Jun 03 13:57:06.378731 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.378532 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-image-registry-private-configuration\") on node \"ip-10-0-134-89.ec2.internal\" DevicePath \"\"" Jun 03 13:57:06.378731 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.378545 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-ca-trust-extracted\") on node \"ip-10-0-134-89.ec2.internal\" DevicePath \"\"" Jun 03 13:57:06.378731 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:06.378556 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-certificates\") on node \"ip-10-0-134-89.ec2.internal\" DevicePath \"\"" Jun 03 13:57:07.185415 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:07.185384 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cd7686cb6-7bhjc" Jun 03 13:57:07.222271 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:07.222240 2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cd7686cb6-7bhjc"] Jun 03 13:57:07.233667 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:07.233644 2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7cd7686cb6-7bhjc"] Jun 03 13:57:07.284198 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:07.284175 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/063b7d6f-5926-4d42-a29c-3ed54eaeb18a-registry-tls\") on node \"ip-10-0-134-89.ec2.internal\" DevicePath \"\"" Jun 03 13:57:08.621698 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:08.621666 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="063b7d6f-5926-4d42-a29c-3ed54eaeb18a" path="/var/lib/kubelet/pods/063b7d6f-5926-4d42-a29c-3ed54eaeb18a/volumes" Jun 03 13:57:09.279212 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:09.279175 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" podUID="a45139c3-0352-430c-acdc-6fe24607c5fc" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 03 13:57:19.278598 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:19.278556 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" podUID="a45139c3-0352-430c-acdc-6fe24607c5fc" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 03 13:57:19.278953 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:19.278639 2565 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" Jun 03 13:57:19.279133 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:19.279102 2565 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"35f94c785fb9183f45dc8755f29fd8a45802bdba26bdf1945603ce14cc93e8ed"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 03 13:57:19.279174 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:19.279153 2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" podUID="a45139c3-0352-430c-acdc-6fe24607c5fc" containerName="service-proxy" containerID="cri-o://35f94c785fb9183f45dc8755f29fd8a45802bdba26bdf1945603ce14cc93e8ed" gracePeriod=30 Jun 03 13:57:20.219085 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:20.219050 2565 generic.go:358] "Generic (PLEG): container finished" podID="a45139c3-0352-430c-acdc-6fe24607c5fc" containerID="35f94c785fb9183f45dc8755f29fd8a45802bdba26bdf1945603ce14cc93e8ed" exitCode=2 Jun 03 13:57:20.219243 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:20.219116 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" event={"ID":"a45139c3-0352-430c-acdc-6fe24607c5fc","Type":"ContainerDied","Data":"35f94c785fb9183f45dc8755f29fd8a45802bdba26bdf1945603ce14cc93e8ed"} Jun 03 13:57:20.219243 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:20.219153 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f786849c7-mjzgv" event={"ID":"a45139c3-0352-430c-acdc-6fe24607c5fc","Type":"ContainerStarted","Data":"a2cc88b94c659d5ac49320363c9b28490a75c8f9a46cc29cd08b4b3e7cd05e52"} Jun 03 13:57:56.456824 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:56.456775 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:57:56.459156 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:56.459131 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/277e2d8b-f5ab-4f08-bb7e-b6981b3932fc-metrics-certs\") pod \"network-metrics-daemon-5m5l8\" (UID: \"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc\") " pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:57:56.622631 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:56.622606 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tj8sj\"" Jun 03 13:57:56.630456 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:56.630431 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m5l8" Jun 03 13:57:56.747650 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:56.747559 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5m5l8"] Jun 03 13:57:56.750130 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:57:56.750104 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277e2d8b_f5ab_4f08_bb7e_b6981b3932fc.slice/crio-56da056960b66f9fd84e5c10db6d654094704d7eac1fcdcf46d80d6651b5b201 WatchSource:0}: Error finding container 56da056960b66f9fd84e5c10db6d654094704d7eac1fcdcf46d80d6651b5b201: Status 404 returned error can't find the container with id 56da056960b66f9fd84e5c10db6d654094704d7eac1fcdcf46d80d6651b5b201 Jun 03 13:57:57.311981 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:57.311926 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5m5l8" event={"ID":"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc","Type":"ContainerStarted","Data":"56da056960b66f9fd84e5c10db6d654094704d7eac1fcdcf46d80d6651b5b201"} Jun 03 13:57:58.315753 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:58.315711 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5m5l8" event={"ID":"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc","Type":"ContainerStarted","Data":"12ff06979ed5ad6435c2f67296183fe48fb2a1dacf9b82d29666d711d5bcc5fa"} Jun 03 13:57:58.315753 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:58.315750 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5m5l8" event={"ID":"277e2d8b-f5ab-4f08-bb7e-b6981b3932fc","Type":"ContainerStarted","Data":"3b30fc7e2583df30381d68af9e554eb9462ceec6bf2b01f441f101b88a1f660c"} Jun 03 13:57:58.332132 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:57:58.332082 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5m5l8" podStartSLOduration=253.352779583 podStartE2EDuration="4m14.332068999s" podCreationTimestamp="2026-06-03 13:53:44 +0000 UTC" firstStartedPulling="2026-06-03 13:57:56.751975251 +0000 UTC m=+252.696619187" lastFinishedPulling="2026-06-03 13:57:57.731264671 +0000 UTC m=+253.675908603" observedRunningTime="2026-06-03 13:57:58.330969538 +0000 UTC m=+254.275613493" watchObservedRunningTime="2026-06-03 13:57:58.332068999 +0000 UTC m=+254.276712969" Jun 03 13:58:25.063800 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:58:25.063755 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" podUID="4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b" Jun 03 13:58:25.063800 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:58:25.063756 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7d2wt" podUID="6724d1a1-68ea-4f1d-a782-7bbaf376213c" Jun 03 13:58:25.381027 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:25.380950 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:58:25.381169 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:25.381130 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7d2wt" Jun 03 13:58:26.066193 ip-10-0-134-89 kubenswrapper[2565]: E0603 13:58:26.066138 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-p66h6" podUID="04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea" Jun 03 13:58:26.382972 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:26.382893 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:58:28.993837 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:28.993789 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:58:28.993837 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:28.993850 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:58:28.996203 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:28.996175 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6724d1a1-68ea-4f1d-a782-7bbaf376213c-metrics-tls\") pod \"dns-default-7d2wt\" (UID: \"6724d1a1-68ea-4f1d-a782-7bbaf376213c\") " pod="openshift-dns/dns-default-7d2wt" Jun 03 13:58:28.996312 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:28.996257 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-9qlnz\" (UID: \"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:58:29.094550 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:29.094521 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:58:29.096809 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:29.096784 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea-cert\") pod \"ingress-canary-p66h6\" (UID: \"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea\") " pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:58:29.284854 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:29.284777 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7fx7r\"" Jun 03 13:58:29.285068 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:29.285048 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zsshc\"" Jun 03 13:58:29.292418 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:29.292396 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7d2wt" Jun 03 13:58:29.292534 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:29.292400 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" Jun 03 13:58:29.386884 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:29.386860 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gxcmr\"" Jun 03 13:58:29.394844 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:29.394816 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p66h6" Jun 03 13:58:29.422121 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:29.422093 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz"] Jun 03 13:58:29.425288 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:58:29.425258 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dfd6ead_68a6_493b_9ad0_37c7c81aaf9b.slice/crio-038554ca40f39bd5154f7672120bc7140ae76df649f1f99d7c8ff673062f0050 WatchSource:0}: Error finding container 038554ca40f39bd5154f7672120bc7140ae76df649f1f99d7c8ff673062f0050: Status 404 returned error can't find the container with id 038554ca40f39bd5154f7672120bc7140ae76df649f1f99d7c8ff673062f0050 Jun 03 13:58:29.446428 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:29.445718 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7d2wt"] Jun 03 13:58:29.448145 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:58:29.448092 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6724d1a1_68ea_4f1d_a782_7bbaf376213c.slice/crio-02b43c02d9a685b6c413a50a43e511ac8cbe0d33cbd71ee51c895006d8461b45 WatchSource:0}: Error finding container 02b43c02d9a685b6c413a50a43e511ac8cbe0d33cbd71ee51c895006d8461b45: Status 404 returned error can't find the container with id 02b43c02d9a685b6c413a50a43e511ac8cbe0d33cbd71ee51c895006d8461b45 Jun 03 13:58:29.524150 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:29.524112 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p66h6"] Jun 03 13:58:29.526787 ip-10-0-134-89 kubenswrapper[2565]: W0603 13:58:29.526763 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04c03b8a_4b1b_4b45_8f1d_cc87a84f87ea.slice/crio-d861ff94fb2a84af20f6309190afd430e552380e8a85710c1f01af4d3a926ca3 WatchSource:0}: Error finding container d861ff94fb2a84af20f6309190afd430e552380e8a85710c1f01af4d3a926ca3: Status 404 returned error can't find the container with id d861ff94fb2a84af20f6309190afd430e552380e8a85710c1f01af4d3a926ca3 Jun 03 13:58:30.397215 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:30.397160 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" event={"ID":"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b","Type":"ContainerStarted","Data":"038554ca40f39bd5154f7672120bc7140ae76df649f1f99d7c8ff673062f0050"} Jun 03 13:58:30.398677 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:30.398643 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p66h6" event={"ID":"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea","Type":"ContainerStarted","Data":"d861ff94fb2a84af20f6309190afd430e552380e8a85710c1f01af4d3a926ca3"} Jun 03 13:58:30.399827 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:30.399721 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7d2wt" event={"ID":"6724d1a1-68ea-4f1d-a782-7bbaf376213c","Type":"ContainerStarted","Data":"02b43c02d9a685b6c413a50a43e511ac8cbe0d33cbd71ee51c895006d8461b45"} Jun 03 13:58:31.404536 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:31.404449 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" event={"ID":"4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b","Type":"ContainerStarted","Data":"e3827cc0b50385aec08754d66d3eb9ab1a1ed2648ce9ed9341b0053b3c315c82"} Jun 03 13:58:31.406290 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:31.406266 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7d2wt" event={"ID":"6724d1a1-68ea-4f1d-a782-7bbaf376213c","Type":"ContainerStarted","Data":"8a98f64c42c24e763462cade8b5cdf6a4facff24077f757cd2fe51f9ad208915"} Jun 03 13:58:31.406290 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:31.406293 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7d2wt" event={"ID":"6724d1a1-68ea-4f1d-a782-7bbaf376213c","Type":"ContainerStarted","Data":"28bc57d310b329bedb309c500225ca0db639c61daacd8fadf5c446a7bba1c95d"} Jun 03 13:58:31.406506 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:31.406446 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7d2wt" Jun 03 13:58:31.426274 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:31.423749 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-9qlnz" podStartSLOduration=270.744643067 podStartE2EDuration="4m32.423732368s" podCreationTimestamp="2026-06-03 13:53:59 +0000 UTC" firstStartedPulling="2026-06-03 13:58:29.429135359 +0000 UTC m=+285.373779292" lastFinishedPulling="2026-06-03 13:58:31.108224658 +0000 UTC m=+287.052868593" observedRunningTime="2026-06-03 13:58:31.422024202 +0000 UTC m=+287.366668158" watchObservedRunningTime="2026-06-03 13:58:31.423732368 +0000 UTC m=+287.368376325" Jun 03 13:58:31.444494 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:31.444437 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-7d2wt" podStartSLOduration=251.782421836 podStartE2EDuration="4m13.444420656s" podCreationTimestamp="2026-06-03 13:54:18 +0000 UTC" firstStartedPulling="2026-06-03 13:58:29.449934125 +0000 UTC m=+285.394578058" lastFinishedPulling="2026-06-03 13:58:31.111932943 +0000 UTC m=+287.056576878" observedRunningTime="2026-06-03 13:58:31.442899785 +0000 UTC m=+287.387543734" watchObservedRunningTime="2026-06-03 13:58:31.444420656 +0000 UTC m=+287.389064613" Jun 03 13:58:32.409908 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:32.409871 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p66h6" event={"ID":"04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea","Type":"ContainerStarted","Data":"333af31bff954d078a2183ae9ed776efc1df6b58d26dd46d15ced4089844804e"} Jun 03 13:58:32.436475 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:32.436425 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p66h6" podStartSLOduration=251.130558699 podStartE2EDuration="4m13.436410028s" podCreationTimestamp="2026-06-03 13:54:19 +0000 UTC" firstStartedPulling="2026-06-03 13:58:29.528557648 +0000 UTC m=+285.473201581" lastFinishedPulling="2026-06-03 13:58:31.834408977 +0000 UTC m=+287.779052910" observedRunningTime="2026-06-03 13:58:32.428870058 +0000 UTC m=+288.373514017" watchObservedRunningTime="2026-06-03 13:58:32.436410028 +0000 UTC m=+288.381053982" Jun 03 13:58:41.411531 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:41.411501 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7d2wt" Jun 03 13:58:44.523351 ip-10-0-134-89 kubenswrapper[2565]: I0603 13:58:44.523315 2565 kubelet.go:1628] "Image garbage collection succeeded" Jun 03 14:00:26.921940 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:26.921900 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9"] Jun 03 14:00:26.924823 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:26.924807 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9" Jun 03 14:00:26.927724 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:26.927704 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 03 14:00:26.928328 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:26.928312 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-tb9tg\"" Jun 03 14:00:26.928440 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:26.928326 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 03 14:00:26.937772 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:26.937748 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9"] Jun 03 14:00:26.986952 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:26.986924 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70a7f4a1-a8c1-4640-9541-48d929750adf-tmp\") pod \"openshift-lws-operator-bfc7f696d-7z2c9\" (UID: \"70a7f4a1-a8c1-4640-9541-48d929750adf\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9" Jun 03 14:00:26.987071 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:26.986963 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg7l9\" (UniqueName: \"kubernetes.io/projected/70a7f4a1-a8c1-4640-9541-48d929750adf-kube-api-access-zg7l9\") pod \"openshift-lws-operator-bfc7f696d-7z2c9\" (UID: \"70a7f4a1-a8c1-4640-9541-48d929750adf\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9" Jun 03 14:00:27.087692 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:27.087658 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70a7f4a1-a8c1-4640-9541-48d929750adf-tmp\") pod \"openshift-lws-operator-bfc7f696d-7z2c9\" (UID: \"70a7f4a1-a8c1-4640-9541-48d929750adf\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9" Jun 03 14:00:27.087838 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:27.087704 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zg7l9\" (UniqueName: \"kubernetes.io/projected/70a7f4a1-a8c1-4640-9541-48d929750adf-kube-api-access-zg7l9\") pod \"openshift-lws-operator-bfc7f696d-7z2c9\" (UID: \"70a7f4a1-a8c1-4640-9541-48d929750adf\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9" Jun 03 14:00:27.088089 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:27.088065 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70a7f4a1-a8c1-4640-9541-48d929750adf-tmp\") pod \"openshift-lws-operator-bfc7f696d-7z2c9\" (UID: \"70a7f4a1-a8c1-4640-9541-48d929750adf\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9" Jun 03 14:00:27.101294 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:27.101260 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg7l9\" (UniqueName: \"kubernetes.io/projected/70a7f4a1-a8c1-4640-9541-48d929750adf-kube-api-access-zg7l9\") pod \"openshift-lws-operator-bfc7f696d-7z2c9\" (UID: \"70a7f4a1-a8c1-4640-9541-48d929750adf\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9" Jun 03 14:00:27.233852 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:27.233803 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9" Jun 03 14:00:27.357452 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:27.357393 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9"] Jun 03 14:00:27.359984 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:00:27.359947 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70a7f4a1_a8c1_4640_9541_48d929750adf.slice/crio-4bb36f55656aaf8d43e0be8ad07d3c056df2d5d2f252c3c614deb3f4982f38b7 WatchSource:0}: Error finding container 4bb36f55656aaf8d43e0be8ad07d3c056df2d5d2f252c3c614deb3f4982f38b7: Status 404 returned error can't find the container with id 4bb36f55656aaf8d43e0be8ad07d3c056df2d5d2f252c3c614deb3f4982f38b7 Jun 03 14:00:27.361440 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:27.361422 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 14:00:27.687002 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:27.686913 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9" event={"ID":"70a7f4a1-a8c1-4640-9541-48d929750adf","Type":"ContainerStarted","Data":"4bb36f55656aaf8d43e0be8ad07d3c056df2d5d2f252c3c614deb3f4982f38b7"} Jun 03 14:00:31.698919 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:31.698823 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9" event={"ID":"70a7f4a1-a8c1-4640-9541-48d929750adf","Type":"ContainerStarted","Data":"4f13d32f027bee9e09fc8040649bc06e48f572971ff01ab5e3d357a46f2729f5"} Jun 03 14:00:31.716438 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:31.716387 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-7z2c9" podStartSLOduration=1.8368440050000001 podStartE2EDuration="5.71637205s" podCreationTimestamp="2026-06-03 14:00:26 +0000 UTC" firstStartedPulling="2026-06-03 14:00:27.361551665 +0000 UTC m=+403.306195598" lastFinishedPulling="2026-06-03 14:00:31.241079706 +0000 UTC m=+407.185723643" observedRunningTime="2026-06-03 14:00:31.715400855 +0000 UTC m=+407.660044812" watchObservedRunningTime="2026-06-03 14:00:31.71637205 +0000 UTC m=+407.661016004" Jun 03 14:00:32.732470 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:32.732439 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-jkwzp"] Jun 03 14:00:32.735400 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:32.735382 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-jkwzp" Jun 03 14:00:32.737924 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:32.737904 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 03 14:00:32.738035 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:32.737907 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 03 14:00:32.738676 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:32.738663 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5mtxl\"" Jun 03 14:00:32.745682 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:32.745661 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-jkwzp"] Jun 03 14:00:32.831861 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:32.831822 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p98vv\" (UniqueName: \"kubernetes.io/projected/b161e771-1f5f-4f95-b6cb-b8614b57596d-kube-api-access-p98vv\") pod \"cert-manager-79c8d999ff-jkwzp\" (UID: \"b161e771-1f5f-4f95-b6cb-b8614b57596d\") " pod="cert-manager/cert-manager-79c8d999ff-jkwzp" Jun 03 14:00:32.832026 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:32.831866 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b161e771-1f5f-4f95-b6cb-b8614b57596d-bound-sa-token\") pod \"cert-manager-79c8d999ff-jkwzp\" (UID: \"b161e771-1f5f-4f95-b6cb-b8614b57596d\") " pod="cert-manager/cert-manager-79c8d999ff-jkwzp" Jun 03 14:00:32.933207 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:32.933178 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p98vv\" (UniqueName: \"kubernetes.io/projected/b161e771-1f5f-4f95-b6cb-b8614b57596d-kube-api-access-p98vv\") pod \"cert-manager-79c8d999ff-jkwzp\" (UID: \"b161e771-1f5f-4f95-b6cb-b8614b57596d\") " pod="cert-manager/cert-manager-79c8d999ff-jkwzp" Jun 03 14:00:32.933331 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:32.933219 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b161e771-1f5f-4f95-b6cb-b8614b57596d-bound-sa-token\") pod \"cert-manager-79c8d999ff-jkwzp\" (UID: \"b161e771-1f5f-4f95-b6cb-b8614b57596d\") " pod="cert-manager/cert-manager-79c8d999ff-jkwzp" Jun 03 14:00:32.952064 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:32.952033 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p98vv\" (UniqueName: \"kubernetes.io/projected/b161e771-1f5f-4f95-b6cb-b8614b57596d-kube-api-access-p98vv\") pod \"cert-manager-79c8d999ff-jkwzp\" (UID: \"b161e771-1f5f-4f95-b6cb-b8614b57596d\") " pod="cert-manager/cert-manager-79c8d999ff-jkwzp" Jun 03 14:00:32.952305 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:32.952287 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b161e771-1f5f-4f95-b6cb-b8614b57596d-bound-sa-token\") pod \"cert-manager-79c8d999ff-jkwzp\" (UID: \"b161e771-1f5f-4f95-b6cb-b8614b57596d\") " pod="cert-manager/cert-manager-79c8d999ff-jkwzp" Jun 03 14:00:33.044106 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:33.044039 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-jkwzp" Jun 03 14:00:33.160947 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:33.160912 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-jkwzp"] Jun 03 14:00:33.163711 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:00:33.163673 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb161e771_1f5f_4f95_b6cb_b8614b57596d.slice/crio-0d73433bf84b214cc4b85aad6ddd674d9ae032922787edc2ad8ddbdc08e23357 WatchSource:0}: Error finding container 0d73433bf84b214cc4b85aad6ddd674d9ae032922787edc2ad8ddbdc08e23357: Status 404 returned error can't find the container with id 0d73433bf84b214cc4b85aad6ddd674d9ae032922787edc2ad8ddbdc08e23357 Jun 03 14:00:33.705257 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:33.705224 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-jkwzp" event={"ID":"b161e771-1f5f-4f95-b6cb-b8614b57596d","Type":"ContainerStarted","Data":"0d73433bf84b214cc4b85aad6ddd674d9ae032922787edc2ad8ddbdc08e23357"} Jun 03 14:00:36.715771 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:36.715730 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-jkwzp" event={"ID":"b161e771-1f5f-4f95-b6cb-b8614b57596d","Type":"ContainerStarted","Data":"b7aed6aae8813591da8c0692bcce7f2b13b9db8f5bad4a246fc011758b1a1f37"} Jun 03 14:00:36.735660 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:36.735617 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-jkwzp" podStartSLOduration=1.366007561 podStartE2EDuration="4.735602087s" podCreationTimestamp="2026-06-03 14:00:32 +0000 UTC" firstStartedPulling="2026-06-03 14:00:33.165623225 +0000 UTC m=+409.110267161" lastFinishedPulling="2026-06-03 14:00:36.535217754 +0000 UTC m=+412.479861687" observedRunningTime="2026-06-03 14:00:36.734572279 +0000 UTC m=+412.679216234" watchObservedRunningTime="2026-06-03 14:00:36.735602087 +0000 UTC m=+412.680246020" Jun 03 14:00:48.110797 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.110767 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp"] Jun 03 14:00:48.117316 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.117296 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.124104 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.124084 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 03 14:00:48.124192 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.124131 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 03 14:00:48.124471 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.124454 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 03 14:00:48.124544 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.124523 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-t5bnl\"" Jun 03 14:00:48.124813 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.124792 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 03 14:00:48.132833 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.132807 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp"] Jun 03 14:00:48.244429 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.244394 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffbvs\" (UniqueName: \"kubernetes.io/projected/5989b631-fc30-42c5-9d33-ce116e0a33a8-kube-api-access-ffbvs\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.244429 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.244431 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5989b631-fc30-42c5-9d33-ce116e0a33a8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.244640 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.244518 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5989b631-fc30-42c5-9d33-ce116e0a33a8-tmp\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.244640 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.244561 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5989b631-fc30-42c5-9d33-ce116e0a33a8-manifests\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.244640 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.244584 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5989b631-fc30-42c5-9d33-ce116e0a33a8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.345262 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.345232 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5989b631-fc30-42c5-9d33-ce116e0a33a8-tmp\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.345262 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.345269 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5989b631-fc30-42c5-9d33-ce116e0a33a8-manifests\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.345500 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.345299 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5989b631-fc30-42c5-9d33-ce116e0a33a8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.345500 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.345483 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ffbvs\" (UniqueName: \"kubernetes.io/projected/5989b631-fc30-42c5-9d33-ce116e0a33a8-kube-api-access-ffbvs\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.345616 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.345523 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5989b631-fc30-42c5-9d33-ce116e0a33a8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.345687 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.345669 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5989b631-fc30-42c5-9d33-ce116e0a33a8-manifests\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.347682 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.347655 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5989b631-fc30-42c5-9d33-ce116e0a33a8-tmp\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.347809 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.347781 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5989b631-fc30-42c5-9d33-ce116e0a33a8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.347969 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.347949 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5989b631-fc30-42c5-9d33-ce116e0a33a8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.360108 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.360085 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffbvs\" (UniqueName: \"kubernetes.io/projected/5989b631-fc30-42c5-9d33-ce116e0a33a8-kube-api-access-ffbvs\") pod \"opendatahub-operator-controller-manager-7c68786f4b-nz2pp\" (UID: \"5989b631-fc30-42c5-9d33-ce116e0a33a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.427641 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.427578 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:48.562512 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.562456 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp"] Jun 03 14:00:48.565243 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:00:48.565212 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5989b631_fc30_42c5_9d33_ce116e0a33a8.slice/crio-a42302e8f807b174074404ddfb29935b27b6eb404869f13329ed79af93d52796 WatchSource:0}: Error finding container a42302e8f807b174074404ddfb29935b27b6eb404869f13329ed79af93d52796: Status 404 returned error can't find the container with id a42302e8f807b174074404ddfb29935b27b6eb404869f13329ed79af93d52796 Jun 03 14:00:48.746918 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:48.746885 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" event={"ID":"5989b631-fc30-42c5-9d33-ce116e0a33a8","Type":"ContainerStarted","Data":"a42302e8f807b174074404ddfb29935b27b6eb404869f13329ed79af93d52796"} Jun 03 14:00:51.758124 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:51.758088 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" event={"ID":"5989b631-fc30-42c5-9d33-ce116e0a33a8","Type":"ContainerStarted","Data":"1521e013097a220181af120f53c5ed7688f3a42311410d435a02fb81fa67122a"} Jun 03 14:00:52.765492 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:52.765455 2565 generic.go:358] "Generic (PLEG): container finished" podID="5989b631-fc30-42c5-9d33-ce116e0a33a8" containerID="1521e013097a220181af120f53c5ed7688f3a42311410d435a02fb81fa67122a" exitCode=0 Jun 03 14:00:52.765897 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:52.765532 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" event={"ID":"5989b631-fc30-42c5-9d33-ce116e0a33a8","Type":"ContainerDied","Data":"1521e013097a220181af120f53c5ed7688f3a42311410d435a02fb81fa67122a"} Jun 03 14:00:53.769574 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:53.769529 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" event={"ID":"5989b631-fc30-42c5-9d33-ce116e0a33a8","Type":"ContainerStarted","Data":"b25d57ec4dec76892c5c95bcc1da8ba5384b54bf53211d77e8fcdc0d41840e96"} Jun 03 14:00:53.769995 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:53.769660 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:00:53.796369 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:00:53.796307 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" podStartSLOduration=1.245768098 podStartE2EDuration="5.796294099s" podCreationTimestamp="2026-06-03 14:00:48 +0000 UTC" firstStartedPulling="2026-06-03 14:00:48.567146522 +0000 UTC m=+424.511790455" lastFinishedPulling="2026-06-03 14:00:53.11767252 +0000 UTC m=+429.062316456" observedRunningTime="2026-06-03 14:00:53.794617985 +0000 UTC m=+429.739261939" watchObservedRunningTime="2026-06-03 14:00:53.796294099 +0000 UTC m=+429.740938054" Jun 03 14:01:04.775234 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:04.775160 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c68786f4b-nz2pp" Jun 03 14:01:07.332185 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:07.332149 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-76588b95-8mtfw"] Jun 03 14:01:07.335254 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:07.335237 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-8mtfw" Jun 03 14:01:07.338452 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:07.338433 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tndlb\"" Jun 03 14:01:07.338966 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:07.338949 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 03 14:01:07.358024 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:07.358003 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-8mtfw"] Jun 03 14:01:07.490417 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:07.490381 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f-cert\") pod \"odh-model-controller-76588b95-8mtfw\" (UID: \"b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f\") " pod="opendatahub/odh-model-controller-76588b95-8mtfw" Jun 03 14:01:07.490570 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:07.490424 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7265h\" (UniqueName: \"kubernetes.io/projected/b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f-kube-api-access-7265h\") pod \"odh-model-controller-76588b95-8mtfw\" (UID: \"b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f\") " pod="opendatahub/odh-model-controller-76588b95-8mtfw" Jun 03 14:01:07.591356 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:07.591277 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f-cert\") pod \"odh-model-controller-76588b95-8mtfw\" (UID: \"b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f\") " pod="opendatahub/odh-model-controller-76588b95-8mtfw" Jun 03 14:01:07.591356 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:07.591319 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7265h\" (UniqueName: \"kubernetes.io/projected/b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f-kube-api-access-7265h\") pod \"odh-model-controller-76588b95-8mtfw\" (UID: \"b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f\") " pod="opendatahub/odh-model-controller-76588b95-8mtfw" Jun 03 14:01:07.591525 ip-10-0-134-89 kubenswrapper[2565]: E0603 14:01:07.591440 2565 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 03 14:01:07.591525 ip-10-0-134-89 kubenswrapper[2565]: E0603 14:01:07.591507 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f-cert podName:b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f nodeName:}" failed. No retries permitted until 2026-06-03 14:01:08.091490669 +0000 UTC m=+444.036134602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f-cert") pod "odh-model-controller-76588b95-8mtfw" (UID: "b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f") : secret "odh-model-controller-webhook-cert" not found Jun 03 14:01:07.616463 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:07.616435 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7265h\" (UniqueName: \"kubernetes.io/projected/b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f-kube-api-access-7265h\") pod \"odh-model-controller-76588b95-8mtfw\" (UID: \"b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f\") " pod="opendatahub/odh-model-controller-76588b95-8mtfw" Jun 03 14:01:08.094405 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.094332 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f-cert\") pod \"odh-model-controller-76588b95-8mtfw\" (UID: \"b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f\") " pod="opendatahub/odh-model-controller-76588b95-8mtfw" Jun 03 14:01:08.096779 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.096754 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f-cert\") pod \"odh-model-controller-76588b95-8mtfw\" (UID: \"b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f\") " pod="opendatahub/odh-model-controller-76588b95-8mtfw" Jun 03 14:01:08.245149 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.245115 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-8mtfw" Jun 03 14:01:08.262288 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.262251 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-6577596cd5-6qwz7"] Jun 03 14:01:08.266645 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.266616 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-6qwz7" Jun 03 14:01:08.269109 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.269085 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 03 14:01:08.278588 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.278563 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-6qwz7"] Jun 03 14:01:08.369943 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.369857 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-8mtfw"] Jun 03 14:01:08.372629 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:01:08.372596 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb135e85a_2c63_4a1e_a8d2_bdd3c29d9b9f.slice/crio-1fb00e5928eb501aeef3edb46686e6dd5e8d99378b3ccf7e1acc96b2f65bcf23 WatchSource:0}: Error finding container 1fb00e5928eb501aeef3edb46686e6dd5e8d99378b3ccf7e1acc96b2f65bcf23: Status 404 returned error can't find the container with id 1fb00e5928eb501aeef3edb46686e6dd5e8d99378b3ccf7e1acc96b2f65bcf23 Jun 03 14:01:08.397159 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.397135 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d24095a-9983-4de2-9130-e6db34032792-tls-certs\") pod \"model-serving-api-6577596cd5-6qwz7\" (UID: \"3d24095a-9983-4de2-9130-e6db34032792\") " pod="opendatahub/model-serving-api-6577596cd5-6qwz7" Jun 03 14:01:08.397248 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.397171 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knkct\" (UniqueName: \"kubernetes.io/projected/3d24095a-9983-4de2-9130-e6db34032792-kube-api-access-knkct\") pod \"model-serving-api-6577596cd5-6qwz7\" (UID: \"3d24095a-9983-4de2-9130-e6db34032792\") " pod="opendatahub/model-serving-api-6577596cd5-6qwz7" Jun 03 14:01:08.497604 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.497569 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d24095a-9983-4de2-9130-e6db34032792-tls-certs\") pod \"model-serving-api-6577596cd5-6qwz7\" (UID: \"3d24095a-9983-4de2-9130-e6db34032792\") " pod="opendatahub/model-serving-api-6577596cd5-6qwz7" Jun 03 14:01:08.497734 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.497624 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-knkct\" (UniqueName: \"kubernetes.io/projected/3d24095a-9983-4de2-9130-e6db34032792-kube-api-access-knkct\") pod \"model-serving-api-6577596cd5-6qwz7\" (UID: \"3d24095a-9983-4de2-9130-e6db34032792\") " pod="opendatahub/model-serving-api-6577596cd5-6qwz7" Jun 03 14:01:08.497734 ip-10-0-134-89 kubenswrapper[2565]: E0603 14:01:08.497708 2565 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 03 14:01:08.497803 ip-10-0-134-89 kubenswrapper[2565]: E0603 14:01:08.497768 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d24095a-9983-4de2-9130-e6db34032792-tls-certs podName:3d24095a-9983-4de2-9130-e6db34032792 nodeName:}" failed. No retries permitted until 2026-06-03 14:01:08.997752582 +0000 UTC m=+444.942396515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/3d24095a-9983-4de2-9130-e6db34032792-tls-certs") pod "model-serving-api-6577596cd5-6qwz7" (UID: "3d24095a-9983-4de2-9130-e6db34032792") : secret "model-serving-api-tls" not found Jun 03 14:01:08.506720 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.506702 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knkct\" (UniqueName: \"kubernetes.io/projected/3d24095a-9983-4de2-9130-e6db34032792-kube-api-access-knkct\") pod \"model-serving-api-6577596cd5-6qwz7\" (UID: \"3d24095a-9983-4de2-9130-e6db34032792\") " pod="opendatahub/model-serving-api-6577596cd5-6qwz7" Jun 03 14:01:08.808692 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:08.808654 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-8mtfw" event={"ID":"b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f","Type":"ContainerStarted","Data":"1fb00e5928eb501aeef3edb46686e6dd5e8d99378b3ccf7e1acc96b2f65bcf23"} Jun 03 14:01:09.001939 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:09.001890 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d24095a-9983-4de2-9130-e6db34032792-tls-certs\") pod \"model-serving-api-6577596cd5-6qwz7\" (UID: \"3d24095a-9983-4de2-9130-e6db34032792\") " pod="opendatahub/model-serving-api-6577596cd5-6qwz7" Jun 03 14:01:09.004969 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:09.004921 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d24095a-9983-4de2-9130-e6db34032792-tls-certs\") pod \"model-serving-api-6577596cd5-6qwz7\" (UID: \"3d24095a-9983-4de2-9130-e6db34032792\") " pod="opendatahub/model-serving-api-6577596cd5-6qwz7" Jun 03 14:01:09.176945 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:09.176849 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-6577596cd5-6qwz7" Jun 03 14:01:09.314596 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:09.314569 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-6577596cd5-6qwz7"] Jun 03 14:01:09.317262 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:01:09.317221 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d24095a_9983_4de2_9130_e6db34032792.slice/crio-6e1daf23b2db028a9e52f24888490c0b80e83e6c9804b37885b66ed061020b52 WatchSource:0}: Error finding container 6e1daf23b2db028a9e52f24888490c0b80e83e6c9804b37885b66ed061020b52: Status 404 returned error can't find the container with id 6e1daf23b2db028a9e52f24888490c0b80e83e6c9804b37885b66ed061020b52 Jun 03 14:01:09.814614 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:09.814579 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-6qwz7" event={"ID":"3d24095a-9983-4de2-9130-e6db34032792","Type":"ContainerStarted","Data":"6e1daf23b2db028a9e52f24888490c0b80e83e6c9804b37885b66ed061020b52"} Jun 03 14:01:12.829286 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:12.829135 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-6577596cd5-6qwz7" event={"ID":"3d24095a-9983-4de2-9130-e6db34032792","Type":"ContainerStarted","Data":"34b99bfe9ab6b41bb75917f7e074f3edfc15fad350553016dd01411109c95892"} Jun 03 14:01:12.829286 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:12.829240 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-6577596cd5-6qwz7" Jun 03 14:01:12.830713 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:12.830688 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-8mtfw" event={"ID":"b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f","Type":"ContainerStarted","Data":"da0f0771f52a6da0ce3bdeeca1a637b289a5acbc6ffe08b120b2ec1dd7278280"} Jun 03 14:01:12.830858 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:12.830814 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-76588b95-8mtfw" Jun 03 14:01:12.849989 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:12.849939 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/model-serving-api-6577596cd5-6qwz7" podStartSLOduration=2.317058011 podStartE2EDuration="4.849924914s" podCreationTimestamp="2026-06-03 14:01:08 +0000 UTC" firstStartedPulling="2026-06-03 14:01:09.319527604 +0000 UTC m=+445.264171548" lastFinishedPulling="2026-06-03 14:01:11.852394503 +0000 UTC m=+447.797038451" observedRunningTime="2026-06-03 14:01:12.847459454 +0000 UTC m=+448.792103410" watchObservedRunningTime="2026-06-03 14:01:12.849924914 +0000 UTC m=+448.794568868" Jun 03 14:01:12.871832 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:12.871787 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-76588b95-8mtfw" podStartSLOduration=1.7503793380000001 podStartE2EDuration="5.871773867s" podCreationTimestamp="2026-06-03 14:01:07 +0000 UTC" firstStartedPulling="2026-06-03 14:01:08.373814214 +0000 UTC m=+444.318458147" lastFinishedPulling="2026-06-03 14:01:12.49520874 +0000 UTC m=+448.439852676" observedRunningTime="2026-06-03 14:01:12.871160592 +0000 UTC m=+448.815804547" watchObservedRunningTime="2026-06-03 14:01:12.871773867 +0000 UTC m=+448.816417822" Jun 03 14:01:17.772499 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.772464 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-x2998"] Jun 03 14:01:17.775662 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.775644 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" Jun 03 14:01:17.786428 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.786407 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-vx52b\"" Jun 03 14:01:17.789398 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.789381 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 03 14:01:17.796249 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.796223 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-x2998"] Jun 03 14:01:17.808181 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.808158 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr"] Jun 03 14:01:17.813161 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.812444 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr" Jun 03 14:01:17.815663 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.815641 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-hfrj5\"" Jun 03 14:01:17.815830 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.815815 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 03 14:01:17.830185 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.830165 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr"] Jun 03 14:01:17.878934 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.878903 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgx2s\" (UniqueName: \"kubernetes.io/projected/ae7117df-a58a-4603-9479-22064dc28838-kube-api-access-kgx2s\") pod \"kserve-controller-manager-7c99ffcdcd-x2998\" (UID: \"ae7117df-a58a-4603-9479-22064dc28838\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" Jun 03 14:01:17.879098 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.878945 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae7117df-a58a-4603-9479-22064dc28838-cert\") pod \"kserve-controller-manager-7c99ffcdcd-x2998\" (UID: \"ae7117df-a58a-4603-9479-22064dc28838\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" Jun 03 14:01:17.912543 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.912502 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-655d7d459-5lvfh"] Jun 03 14:01:17.915611 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.915593 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" Jun 03 14:01:17.919958 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.919934 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 03 14:01:17.920059 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.919934 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 03 14:01:17.920666 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.920632 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 03 14:01:17.920780 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.920767 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 03 14:01:17.928252 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.928232 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-655d7d459-5lvfh"] Jun 03 14:01:17.979711 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.979679 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mglx9\" (UniqueName: \"kubernetes.io/projected/0c76e725-da9b-4032-a218-f723626f5402-kube-api-access-mglx9\") pod \"llmisvc-controller-manager-c6b9847d7-7txxr\" (UID: \"0c76e725-da9b-4032-a218-f723626f5402\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr" Jun 03 14:01:17.979881 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.979714 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c76e725-da9b-4032-a218-f723626f5402-cert\") pod \"llmisvc-controller-manager-c6b9847d7-7txxr\" (UID: \"0c76e725-da9b-4032-a218-f723626f5402\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr" Jun 03 14:01:17.979881 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.979850 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kgx2s\" (UniqueName: \"kubernetes.io/projected/ae7117df-a58a-4603-9479-22064dc28838-kube-api-access-kgx2s\") pod \"kserve-controller-manager-7c99ffcdcd-x2998\" (UID: \"ae7117df-a58a-4603-9479-22064dc28838\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" Jun 03 14:01:17.980005 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.979899 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae7117df-a58a-4603-9479-22064dc28838-cert\") pod \"kserve-controller-manager-7c99ffcdcd-x2998\" (UID: \"ae7117df-a58a-4603-9479-22064dc28838\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" Jun 03 14:01:17.980058 ip-10-0-134-89 kubenswrapper[2565]: E0603 14:01:17.980031 2565 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jun 03 14:01:17.980109 ip-10-0-134-89 kubenswrapper[2565]: E0603 14:01:17.980098 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae7117df-a58a-4603-9479-22064dc28838-cert podName:ae7117df-a58a-4603-9479-22064dc28838 nodeName:}" failed. No retries permitted until 2026-06-03 14:01:18.480076438 +0000 UTC m=+454.424720383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ae7117df-a58a-4603-9479-22064dc28838-cert") pod "kserve-controller-manager-7c99ffcdcd-x2998" (UID: "ae7117df-a58a-4603-9479-22064dc28838") : secret "kserve-webhook-server-cert" not found Jun 03 14:01:17.999175 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:17.999148 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgx2s\" (UniqueName: \"kubernetes.io/projected/ae7117df-a58a-4603-9479-22064dc28838-kube-api-access-kgx2s\") pod \"kserve-controller-manager-7c99ffcdcd-x2998\" (UID: \"ae7117df-a58a-4603-9479-22064dc28838\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" Jun 03 14:01:18.080426 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.080300 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c76e725-da9b-4032-a218-f723626f5402-cert\") pod \"llmisvc-controller-manager-c6b9847d7-7txxr\" (UID: \"0c76e725-da9b-4032-a218-f723626f5402\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr" Jun 03 14:01:18.080426 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.080410 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b06dd9c5-058e-4bb8-913d-481be4b8e92d-tmp\") pod \"kube-auth-proxy-655d7d459-5lvfh\" (UID: \"b06dd9c5-058e-4bb8-913d-481be4b8e92d\") " pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" Jun 03 14:01:18.080621 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.080446 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b06dd9c5-058e-4bb8-913d-481be4b8e92d-tls-certs\") pod \"kube-auth-proxy-655d7d459-5lvfh\" (UID: \"b06dd9c5-058e-4bb8-913d-481be4b8e92d\") " pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" Jun 03 14:01:18.080621 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.080484 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7tr4\" (UniqueName: \"kubernetes.io/projected/b06dd9c5-058e-4bb8-913d-481be4b8e92d-kube-api-access-x7tr4\") pod \"kube-auth-proxy-655d7d459-5lvfh\" (UID: \"b06dd9c5-058e-4bb8-913d-481be4b8e92d\") " pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" Jun 03 14:01:18.080689 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.080614 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mglx9\" (UniqueName: \"kubernetes.io/projected/0c76e725-da9b-4032-a218-f723626f5402-kube-api-access-mglx9\") pod \"llmisvc-controller-manager-c6b9847d7-7txxr\" (UID: \"0c76e725-da9b-4032-a218-f723626f5402\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr" Jun 03 14:01:18.082797 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.082769 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c76e725-da9b-4032-a218-f723626f5402-cert\") pod \"llmisvc-controller-manager-c6b9847d7-7txxr\" (UID: \"0c76e725-da9b-4032-a218-f723626f5402\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr" Jun 03 14:01:18.102194 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.102162 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mglx9\" (UniqueName: \"kubernetes.io/projected/0c76e725-da9b-4032-a218-f723626f5402-kube-api-access-mglx9\") pod \"llmisvc-controller-manager-c6b9847d7-7txxr\" (UID: \"0c76e725-da9b-4032-a218-f723626f5402\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr" Jun 03 14:01:18.122888 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.122868 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr" Jun 03 14:01:18.181143 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.181086 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b06dd9c5-058e-4bb8-913d-481be4b8e92d-tmp\") pod \"kube-auth-proxy-655d7d459-5lvfh\" (UID: \"b06dd9c5-058e-4bb8-913d-481be4b8e92d\") " pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" Jun 03 14:01:18.181143 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.181142 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b06dd9c5-058e-4bb8-913d-481be4b8e92d-tls-certs\") pod \"kube-auth-proxy-655d7d459-5lvfh\" (UID: \"b06dd9c5-058e-4bb8-913d-481be4b8e92d\") " pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" Jun 03 14:01:18.181463 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.181189 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x7tr4\" (UniqueName: \"kubernetes.io/projected/b06dd9c5-058e-4bb8-913d-481be4b8e92d-kube-api-access-x7tr4\") pod \"kube-auth-proxy-655d7d459-5lvfh\" (UID: \"b06dd9c5-058e-4bb8-913d-481be4b8e92d\") " pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" Jun 03 14:01:18.183877 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.183812 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b06dd9c5-058e-4bb8-913d-481be4b8e92d-tmp\") pod \"kube-auth-proxy-655d7d459-5lvfh\" (UID: \"b06dd9c5-058e-4bb8-913d-481be4b8e92d\") " pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" Jun 03 14:01:18.184867 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.184838 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b06dd9c5-058e-4bb8-913d-481be4b8e92d-tls-certs\") pod \"kube-auth-proxy-655d7d459-5lvfh\" (UID: \"b06dd9c5-058e-4bb8-913d-481be4b8e92d\") " pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" Jun 03 14:01:18.204928 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.204904 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7tr4\" (UniqueName: \"kubernetes.io/projected/b06dd9c5-058e-4bb8-913d-481be4b8e92d-kube-api-access-x7tr4\") pod \"kube-auth-proxy-655d7d459-5lvfh\" (UID: \"b06dd9c5-058e-4bb8-913d-481be4b8e92d\") " pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" Jun 03 14:01:18.224721 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.224692 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" Jun 03 14:01:18.291699 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.291676 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr"] Jun 03 14:01:18.293620 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:01:18.293589 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0c76e725_da9b_4032_a218_f723626f5402.slice/crio-c1d3d5a18e78448df736a1bc17c5edcc1563b2900bf9123d21053b89fd9c8170 WatchSource:0}: Error finding container c1d3d5a18e78448df736a1bc17c5edcc1563b2900bf9123d21053b89fd9c8170: Status 404 returned error can't find the container with id c1d3d5a18e78448df736a1bc17c5edcc1563b2900bf9123d21053b89fd9c8170 Jun 03 14:01:18.356039 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.356016 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-655d7d459-5lvfh"] Jun 03 14:01:18.358367 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:01:18.358323 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb06dd9c5_058e_4bb8_913d_481be4b8e92d.slice/crio-220a9e48a37bfd235c8bc1ee513bec920c4dfd96a9ad1b5f87465504e910ea15 WatchSource:0}: Error finding container 220a9e48a37bfd235c8bc1ee513bec920c4dfd96a9ad1b5f87465504e910ea15: Status 404 returned error can't find the container with id 220a9e48a37bfd235c8bc1ee513bec920c4dfd96a9ad1b5f87465504e910ea15 Jun 03 14:01:18.484252 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.484223 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae7117df-a58a-4603-9479-22064dc28838-cert\") pod \"kserve-controller-manager-7c99ffcdcd-x2998\" (UID: \"ae7117df-a58a-4603-9479-22064dc28838\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" Jun 03 14:01:18.486750 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.486730 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae7117df-a58a-4603-9479-22064dc28838-cert\") pod \"kserve-controller-manager-7c99ffcdcd-x2998\" (UID: \"ae7117df-a58a-4603-9479-22064dc28838\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" Jun 03 14:01:18.685083 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.685000 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" Jun 03 14:01:18.828014 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.827976 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-x2998"] Jun 03 14:01:18.830889 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:01:18.830854 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae7117df_a58a_4603_9479_22064dc28838.slice/crio-3a5210c5b6d04c3a6902bbd561dd91e308ddc3ecee2557cbb4071e36fa3dca1f WatchSource:0}: Error finding container 3a5210c5b6d04c3a6902bbd561dd91e308ddc3ecee2557cbb4071e36fa3dca1f: Status 404 returned error can't find the container with id 3a5210c5b6d04c3a6902bbd561dd91e308ddc3ecee2557cbb4071e36fa3dca1f Jun 03 14:01:18.849274 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.849237 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" event={"ID":"ae7117df-a58a-4603-9479-22064dc28838","Type":"ContainerStarted","Data":"3a5210c5b6d04c3a6902bbd561dd91e308ddc3ecee2557cbb4071e36fa3dca1f"} Jun 03 14:01:18.850350 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.850308 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" event={"ID":"b06dd9c5-058e-4bb8-913d-481be4b8e92d","Type":"ContainerStarted","Data":"220a9e48a37bfd235c8bc1ee513bec920c4dfd96a9ad1b5f87465504e910ea15"} Jun 03 14:01:18.851438 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:18.851412 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr" event={"ID":"0c76e725-da9b-4032-a218-f723626f5402","Type":"ContainerStarted","Data":"c1d3d5a18e78448df736a1bc17c5edcc1563b2900bf9123d21053b89fd9c8170"} Jun 03 14:01:22.870680 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:22.870597 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr" event={"ID":"0c76e725-da9b-4032-a218-f723626f5402","Type":"ContainerStarted","Data":"4c73b0a9f4bdf602225e850cf3a391acc9b8bbb4cd53974d103b1dd7132855d5"} Jun 03 14:01:22.871119 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:22.870688 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr" Jun 03 14:01:22.872011 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:22.871981 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" event={"ID":"ae7117df-a58a-4603-9479-22064dc28838","Type":"ContainerStarted","Data":"1a9fda762c06c15e29b66d7ab3e82bc5b1e5a46ccb3a043fbfa26ee84adfbbd1"} Jun 03 14:01:22.872147 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:22.872117 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" Jun 03 14:01:22.873363 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:22.873323 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" event={"ID":"b06dd9c5-058e-4bb8-913d-481be4b8e92d","Type":"ContainerStarted","Data":"569a3bf44509b602ae7be2b308cf0be5bd0375207167a5232f80ce5b41a58084"} Jun 03 14:01:22.895951 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:22.895908 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr" podStartSLOduration=1.626781429 podStartE2EDuration="5.895897709s" podCreationTimestamp="2026-06-03 14:01:17 +0000 UTC" firstStartedPulling="2026-06-03 14:01:18.294883571 +0000 UTC m=+454.239527504" lastFinishedPulling="2026-06-03 14:01:22.563999847 +0000 UTC m=+458.508643784" observedRunningTime="2026-06-03 14:01:22.893866688 +0000 UTC m=+458.838510642" watchObservedRunningTime="2026-06-03 14:01:22.895897709 +0000 UTC m=+458.840541701" Jun 03 14:01:22.920442 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:22.920389 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-655d7d459-5lvfh" podStartSLOduration=1.7191186539999999 podStartE2EDuration="5.920374603s" podCreationTimestamp="2026-06-03 14:01:17 +0000 UTC" firstStartedPulling="2026-06-03 14:01:18.360024672 +0000 UTC m=+454.304668608" lastFinishedPulling="2026-06-03 14:01:22.561280624 +0000 UTC m=+458.505924557" observedRunningTime="2026-06-03 14:01:22.920135161 +0000 UTC m=+458.864779117" watchObservedRunningTime="2026-06-03 14:01:22.920374603 +0000 UTC m=+458.865018553" Jun 03 14:01:22.952925 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:22.952865 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" podStartSLOduration=2.219598871 podStartE2EDuration="5.952846402s" podCreationTimestamp="2026-06-03 14:01:17 +0000 UTC" firstStartedPulling="2026-06-03 14:01:18.832517833 +0000 UTC m=+454.777161765" lastFinishedPulling="2026-06-03 14:01:22.565765349 +0000 UTC m=+458.510409296" observedRunningTime="2026-06-03 14:01:22.951579139 +0000 UTC m=+458.896223093" watchObservedRunningTime="2026-06-03 14:01:22.952846402 +0000 UTC m=+458.897490360" Jun 03 14:01:23.836134 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:23.836102 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-76588b95-8mtfw" Jun 03 14:01:23.839323 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:23.839301 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-6577596cd5-6qwz7" Jun 03 14:01:33.639237 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.639204 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-7cffc"] Jun 03 14:01:33.642146 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.642131 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-7cffc" Jun 03 14:01:33.644674 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.644656 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 03 14:01:33.644796 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.644690 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 03 14:01:33.645020 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.645007 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-gr2wb\"" Jun 03 14:01:33.652914 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.652889 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-7cffc"] Jun 03 14:01:33.708182 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.708154 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s8cd\" (UniqueName: \"kubernetes.io/projected/756813cb-2387-4cc5-87c2-3f5cd9d80fd6-kube-api-access-4s8cd\") pod \"servicemesh-operator3-55f49c5f94-7cffc\" (UID: \"756813cb-2387-4cc5-87c2-3f5cd9d80fd6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-7cffc" Jun 03 14:01:33.708324 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.708194 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/756813cb-2387-4cc5-87c2-3f5cd9d80fd6-operator-config\") pod \"servicemesh-operator3-55f49c5f94-7cffc\" (UID: \"756813cb-2387-4cc5-87c2-3f5cd9d80fd6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-7cffc" Jun 03 14:01:33.809201 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.809168 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4s8cd\" (UniqueName: \"kubernetes.io/projected/756813cb-2387-4cc5-87c2-3f5cd9d80fd6-kube-api-access-4s8cd\") pod \"servicemesh-operator3-55f49c5f94-7cffc\" (UID: \"756813cb-2387-4cc5-87c2-3f5cd9d80fd6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-7cffc" Jun 03 14:01:33.809201 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.809203 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/756813cb-2387-4cc5-87c2-3f5cd9d80fd6-operator-config\") pod \"servicemesh-operator3-55f49c5f94-7cffc\" (UID: \"756813cb-2387-4cc5-87c2-3f5cd9d80fd6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-7cffc" Jun 03 14:01:33.811783 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.811752 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/756813cb-2387-4cc5-87c2-3f5cd9d80fd6-operator-config\") pod \"servicemesh-operator3-55f49c5f94-7cffc\" (UID: \"756813cb-2387-4cc5-87c2-3f5cd9d80fd6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-7cffc" Jun 03 14:01:33.825292 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.825269 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s8cd\" (UniqueName: \"kubernetes.io/projected/756813cb-2387-4cc5-87c2-3f5cd9d80fd6-kube-api-access-4s8cd\") pod \"servicemesh-operator3-55f49c5f94-7cffc\" (UID: \"756813cb-2387-4cc5-87c2-3f5cd9d80fd6\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-7cffc" Jun 03 14:01:33.898472 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.898397 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-549d6557c8-fkfmq"] Jun 03 14:01:33.901654 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.901635 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-549d6557c8-fkfmq" Jun 03 14:01:33.903974 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.903952 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5gbsr\"" Jun 03 14:01:33.904449 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.904431 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 03 14:01:33.917358 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.917320 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-549d6557c8-fkfmq"] Jun 03 14:01:33.951135 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:33.951106 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-7cffc" Jun 03 14:01:34.010479 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:34.010449 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/7fe4b8fe-65f0-4b05-8690-d2469d0651d7-redirect-config\") pod \"dashboard-redirect-549d6557c8-fkfmq\" (UID: \"7fe4b8fe-65f0-4b05-8690-d2469d0651d7\") " pod="opendatahub/dashboard-redirect-549d6557c8-fkfmq" Jun 03 14:01:34.010621 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:34.010504 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7hpg\" (UniqueName: \"kubernetes.io/projected/7fe4b8fe-65f0-4b05-8690-d2469d0651d7-kube-api-access-m7hpg\") pod \"dashboard-redirect-549d6557c8-fkfmq\" (UID: \"7fe4b8fe-65f0-4b05-8690-d2469d0651d7\") " pod="opendatahub/dashboard-redirect-549d6557c8-fkfmq" Jun 03 14:01:34.081508 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:34.081482 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-7cffc"] Jun 03 14:01:34.084349 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:01:34.084316 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod756813cb_2387_4cc5_87c2_3f5cd9d80fd6.slice/crio-0f63c8efdfb6418854fd6b7cc3efa5c1a4fc48fd4bbcd83978c3a579163d5beb WatchSource:0}: Error finding container 0f63c8efdfb6418854fd6b7cc3efa5c1a4fc48fd4bbcd83978c3a579163d5beb: Status 404 returned error can't find the container with id 0f63c8efdfb6418854fd6b7cc3efa5c1a4fc48fd4bbcd83978c3a579163d5beb Jun 03 14:01:34.111851 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:34.111820 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/7fe4b8fe-65f0-4b05-8690-d2469d0651d7-redirect-config\") pod \"dashboard-redirect-549d6557c8-fkfmq\" (UID: \"7fe4b8fe-65f0-4b05-8690-d2469d0651d7\") " pod="opendatahub/dashboard-redirect-549d6557c8-fkfmq" Jun 03 14:01:34.111957 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:34.111887 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m7hpg\" (UniqueName: \"kubernetes.io/projected/7fe4b8fe-65f0-4b05-8690-d2469d0651d7-kube-api-access-m7hpg\") pod \"dashboard-redirect-549d6557c8-fkfmq\" (UID: \"7fe4b8fe-65f0-4b05-8690-d2469d0651d7\") " pod="opendatahub/dashboard-redirect-549d6557c8-fkfmq" Jun 03 14:01:34.112466 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:34.112447 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/7fe4b8fe-65f0-4b05-8690-d2469d0651d7-redirect-config\") pod \"dashboard-redirect-549d6557c8-fkfmq\" (UID: \"7fe4b8fe-65f0-4b05-8690-d2469d0651d7\") " pod="opendatahub/dashboard-redirect-549d6557c8-fkfmq" Jun 03 14:01:34.120852 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:34.120833 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7hpg\" (UniqueName: \"kubernetes.io/projected/7fe4b8fe-65f0-4b05-8690-d2469d0651d7-kube-api-access-m7hpg\") pod \"dashboard-redirect-549d6557c8-fkfmq\" (UID: \"7fe4b8fe-65f0-4b05-8690-d2469d0651d7\") " pod="opendatahub/dashboard-redirect-549d6557c8-fkfmq" Jun 03 14:01:34.212052 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:34.212019 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-549d6557c8-fkfmq" Jun 03 14:01:34.343121 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:34.343035 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-549d6557c8-fkfmq"] Jun 03 14:01:34.345163 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:01:34.345140 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fe4b8fe_65f0_4b05_8690_d2469d0651d7.slice/crio-54f877cea8fbb4811a0cc549af9bd46c6354e03de1ca438def3ae7bd508dbc48 WatchSource:0}: Error finding container 54f877cea8fbb4811a0cc549af9bd46c6354e03de1ca438def3ae7bd508dbc48: Status 404 returned error can't find the container with id 54f877cea8fbb4811a0cc549af9bd46c6354e03de1ca438def3ae7bd508dbc48 Jun 03 14:01:34.911264 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:34.911226 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-7cffc" event={"ID":"756813cb-2387-4cc5-87c2-3f5cd9d80fd6","Type":"ContainerStarted","Data":"0f63c8efdfb6418854fd6b7cc3efa5c1a4fc48fd4bbcd83978c3a579163d5beb"} Jun 03 14:01:34.912095 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:34.912070 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-549d6557c8-fkfmq" event={"ID":"7fe4b8fe-65f0-4b05-8690-d2469d0651d7","Type":"ContainerStarted","Data":"54f877cea8fbb4811a0cc549af9bd46c6354e03de1ca438def3ae7bd508dbc48"} Jun 03 14:01:36.923939 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:36.923139 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-7cffc" event={"ID":"756813cb-2387-4cc5-87c2-3f5cd9d80fd6","Type":"ContainerStarted","Data":"830e1025609b7b4fbf781023f2a96d4b3600304a50466a5d205e40c934e24a42"} Jun 03 14:01:36.923939 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:36.923900 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-7cffc" Jun 03 14:01:36.957269 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:36.957205 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-7cffc" podStartSLOduration=1.307386752 podStartE2EDuration="3.957186931s" podCreationTimestamp="2026-06-03 14:01:33 +0000 UTC" firstStartedPulling="2026-06-03 14:01:34.087582019 +0000 UTC m=+470.032225952" lastFinishedPulling="2026-06-03 14:01:36.737382191 +0000 UTC m=+472.682026131" observedRunningTime="2026-06-03 14:01:36.954490072 +0000 UTC m=+472.899134028" watchObservedRunningTime="2026-06-03 14:01:36.957186931 +0000 UTC m=+472.901830886" Jun 03 14:01:40.942017 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:40.941927 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-549d6557c8-fkfmq" event={"ID":"7fe4b8fe-65f0-4b05-8690-d2469d0651d7","Type":"ContainerStarted","Data":"fc30dcdcf419bbbb678bd4e4ec4aa041ac2422e263864abef0561736bd63614e"} Jun 03 14:01:40.961748 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:40.961681 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-549d6557c8-fkfmq" podStartSLOduration=1.666229593 podStartE2EDuration="7.961662979s" podCreationTimestamp="2026-06-03 14:01:33 +0000 UTC" firstStartedPulling="2026-06-03 14:01:34.34646454 +0000 UTC m=+470.291108474" lastFinishedPulling="2026-06-03 14:01:40.641897925 +0000 UTC m=+476.586541860" observedRunningTime="2026-06-03 14:01:40.960654113 +0000 UTC m=+476.905298080" watchObservedRunningTime="2026-06-03 14:01:40.961662979 +0000 UTC m=+476.906306936" Jun 03 14:01:44.390846 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.390805 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw"] Jun 03 14:01:44.428113 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.428079 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw"] Jun 03 14:01:44.428267 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.428206 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.430981 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.430958 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 03 14:01:44.431129 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.430967 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gn87s\"" Jun 03 14:01:44.431129 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.431110 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 03 14:01:44.431488 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.431473 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 03 14:01:44.432039 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.432025 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 03 14:01:44.499540 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.499505 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-local-certs\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.499703 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.499547 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-cacerts\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.499703 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.499576 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.499703 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.499653 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-istio-token\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.499807 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.499710 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.499807 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.499753 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.499807 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.499769 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6jhc\" (UniqueName: \"kubernetes.io/projected/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-kube-api-access-z6jhc\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.600906 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.600879 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.601125 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.600910 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z6jhc\" (UniqueName: \"kubernetes.io/projected/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-kube-api-access-z6jhc\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.601125 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.600933 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-local-certs\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.601125 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.600958 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-cacerts\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.601125 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.600984 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.601125 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.601027 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-istio-token\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.601125 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.601072 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.603209 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.603184 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-local-certs\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.605422 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.605404 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 03 14:01:44.605880 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.605854 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 03 14:01:44.614563 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.614527 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-cacerts\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.615166 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.615142 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6jhc\" (UniqueName: \"kubernetes.io/projected/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-kube-api-access-z6jhc\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.615648 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.615587 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 03 14:01:44.616384 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.616365 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.617619 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.617595 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-istio-token\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.618478 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.618454 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 03 14:01:44.621976 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.621953 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.624100 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.624079 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a74827eb-4b95-4cf6-9f34-fc17310ddf7d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-km2pw\" (UID: \"a74827eb-4b95-4cf6-9f34-fc17310ddf7d\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.740857 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.740825 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gn87s\"" Jun 03 14:01:44.749100 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.749068 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:44.884110 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.883815 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw"] Jun 03 14:01:44.886654 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:01:44.886626 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda74827eb_4b95_4cf6_9f34_fc17310ddf7d.slice/crio-387b874c1ff25ebb5e8aad8758dd93d811604524b352a75b49da1e2cf3e1d641 WatchSource:0}: Error finding container 387b874c1ff25ebb5e8aad8758dd93d811604524b352a75b49da1e2cf3e1d641: Status 404 returned error can't find the container with id 387b874c1ff25ebb5e8aad8758dd93d811604524b352a75b49da1e2cf3e1d641 Jun 03 14:01:44.955263 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:44.955226 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" event={"ID":"a74827eb-4b95-4cf6-9f34-fc17310ddf7d","Type":"ContainerStarted","Data":"387b874c1ff25ebb5e8aad8758dd93d811604524b352a75b49da1e2cf3e1d641"} Jun 03 14:01:47.376785 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:47.376736 2565 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 03 14:01:47.377065 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:47.376822 2565 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 03 14:01:47.967680 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:47.967646 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" event={"ID":"a74827eb-4b95-4cf6-9f34-fc17310ddf7d","Type":"ContainerStarted","Data":"a5d2e2fa6fb1e19e4a180a7e880fb793d77a47694b38382957eb290cdb57e071"} Jun 03 14:01:47.967834 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:47.967767 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:47.989968 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:47.989916 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" podStartSLOduration=1.5021152949999999 podStartE2EDuration="3.989900604s" podCreationTimestamp="2026-06-03 14:01:44 +0000 UTC" firstStartedPulling="2026-06-03 14:01:44.888722847 +0000 UTC m=+480.833366783" lastFinishedPulling="2026-06-03 14:01:47.376508158 +0000 UTC m=+483.321152092" observedRunningTime="2026-06-03 14:01:47.986960347 +0000 UTC m=+483.931604302" watchObservedRunningTime="2026-06-03 14:01:47.989900604 +0000 UTC m=+483.934544563" Jun 03 14:01:48.936841 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:48.936803 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-7cffc" Jun 03 14:01:48.972939 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:48.972908 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-km2pw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 03 14:01:48.973100 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:48.972955 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" podUID="a74827eb-4b95-4cf6-9f34-fc17310ddf7d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 03 14:01:51.972117 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:51.972078 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-km2pw" Jun 03 14:01:53.879403 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:53.879368 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-7txxr" Jun 03 14:01:53.882463 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:01:53.882438 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-x2998" Jun 03 14:03:13.464755 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.464722 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl"] Jun 03 14:03:13.468086 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.468064 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" Jun 03 14:03:13.472142 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.472119 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nzlkl\"" Jun 03 14:03:13.472252 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.472138 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 03 14:03:13.472252 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.472138 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 03 14:03:13.489268 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.489246 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl"] Jun 03 14:03:13.614585 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.614554 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgkwc\" (UniqueName: \"kubernetes.io/projected/365c4e39-df7e-437e-a3ba-cc5c2282c650-kube-api-access-vgkwc\") pod \"kuadrant-operator-controller-manager-84b657d985-ctzrl\" (UID: \"365c4e39-df7e-437e-a3ba-cc5c2282c650\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" Jun 03 14:03:13.614729 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.614598 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/365c4e39-df7e-437e-a3ba-cc5c2282c650-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-ctzrl\" (UID: \"365c4e39-df7e-437e-a3ba-cc5c2282c650\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" Jun 03 14:03:13.715691 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.715615 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vgkwc\" (UniqueName: \"kubernetes.io/projected/365c4e39-df7e-437e-a3ba-cc5c2282c650-kube-api-access-vgkwc\") pod \"kuadrant-operator-controller-manager-84b657d985-ctzrl\" (UID: \"365c4e39-df7e-437e-a3ba-cc5c2282c650\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" Jun 03 14:03:13.715691 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.715660 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/365c4e39-df7e-437e-a3ba-cc5c2282c650-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-ctzrl\" (UID: \"365c4e39-df7e-437e-a3ba-cc5c2282c650\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" Jun 03 14:03:13.716101 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.716080 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/365c4e39-df7e-437e-a3ba-cc5c2282c650-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-ctzrl\" (UID: \"365c4e39-df7e-437e-a3ba-cc5c2282c650\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" Jun 03 14:03:13.731028 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.731003 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgkwc\" (UniqueName: \"kubernetes.io/projected/365c4e39-df7e-437e-a3ba-cc5c2282c650-kube-api-access-vgkwc\") pod \"kuadrant-operator-controller-manager-84b657d985-ctzrl\" (UID: \"365c4e39-df7e-437e-a3ba-cc5c2282c650\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" Jun 03 14:03:13.777441 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.777416 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" Jun 03 14:03:13.917645 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.917623 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl"] Jun 03 14:03:13.919821 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:03:13.919785 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod365c4e39_df7e_437e_a3ba_cc5c2282c650.slice/crio-00c20ddbdd1ae4dab97bb8b01718e5c4c0e066864a5bd82df01465374278c4df WatchSource:0}: Error finding container 00c20ddbdd1ae4dab97bb8b01718e5c4c0e066864a5bd82df01465374278c4df: Status 404 returned error can't find the container with id 00c20ddbdd1ae4dab97bb8b01718e5c4c0e066864a5bd82df01465374278c4df Jun 03 14:03:13.964232 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.964199 2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl"] Jun 03 14:03:13.985905 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:13.985833 2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl"] Jun 03 14:03:14.039872 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:14.039843 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-l2x47"] Jun 03 14:03:14.044327 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:14.044310 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-l2x47" Jun 03 14:03:14.046755 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:14.046731 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-45bh4\"" Jun 03 14:03:14.058196 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:14.058171 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-l2x47"] Jun 03 14:03:14.119465 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:14.119432 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gfmr\" (UniqueName: \"kubernetes.io/projected/38d10a07-2d15-4089-9346-39afc5d371f7-kube-api-access-4gfmr\") pod \"limitador-operator-controller-manager-85c4996f8c-l2x47\" (UID: \"38d10a07-2d15-4089-9346-39afc5d371f7\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-l2x47" Jun 03 14:03:14.220786 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:14.220755 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4gfmr\" (UniqueName: \"kubernetes.io/projected/38d10a07-2d15-4089-9346-39afc5d371f7-kube-api-access-4gfmr\") pod \"limitador-operator-controller-manager-85c4996f8c-l2x47\" (UID: \"38d10a07-2d15-4089-9346-39afc5d371f7\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-l2x47" Jun 03 14:03:14.231586 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:14.231566 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gfmr\" (UniqueName: \"kubernetes.io/projected/38d10a07-2d15-4089-9346-39afc5d371f7-kube-api-access-4gfmr\") pod \"limitador-operator-controller-manager-85c4996f8c-l2x47\" (UID: \"38d10a07-2d15-4089-9346-39afc5d371f7\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-l2x47" Jun 03 14:03:14.354826 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:14.354744 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-l2x47" Jun 03 14:03:14.500291 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:14.500264 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-l2x47"] Jun 03 14:03:14.501194 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:03:14.501168 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38d10a07_2d15_4089_9346_39afc5d371f7.slice/crio-bf38fa7140394fbfb7be797903ae1665614b335c168a7ae84d8fa8cd190845dc WatchSource:0}: Error finding container bf38fa7140394fbfb7be797903ae1665614b335c168a7ae84d8fa8cd190845dc: Status 404 returned error can't find the container with id bf38fa7140394fbfb7be797903ae1665614b335c168a7ae84d8fa8cd190845dc Jun 03 14:03:15.245271 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:15.245239 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-l2x47" event={"ID":"38d10a07-2d15-4089-9346-39afc5d371f7","Type":"ContainerStarted","Data":"bf38fa7140394fbfb7be797903ae1665614b335c168a7ae84d8fa8cd190845dc"} Jun 03 14:03:17.255979 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:17.255940 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-l2x47" event={"ID":"38d10a07-2d15-4089-9346-39afc5d371f7","Type":"ContainerStarted","Data":"d856488b42cc446e048f43430bb20b1a8575f13d312521398380e3c550940897"} Jun 03 14:03:17.256403 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:17.256095 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-l2x47" Jun 03 14:03:17.307277 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:17.307218 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-l2x47" podStartSLOduration=1.311204369 podStartE2EDuration="3.307203124s" podCreationTimestamp="2026-06-03 14:03:14 +0000 UTC" firstStartedPulling="2026-06-03 14:03:14.503104181 +0000 UTC m=+570.447748114" lastFinishedPulling="2026-06-03 14:03:16.499102934 +0000 UTC m=+572.443746869" observedRunningTime="2026-06-03 14:03:17.30441633 +0000 UTC m=+573.249060284" watchObservedRunningTime="2026-06-03 14:03:17.307203124 +0000 UTC m=+573.251847115" Jun 03 14:03:19.263910 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:19.263872 2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" podUID="365c4e39-df7e-437e-a3ba-cc5c2282c650" containerName="manager" containerID="cri-o://a3e51b55337bad11367163089026698eddfa13b8cefc5b9280b58c015258f341" gracePeriod=2 Jun 03 14:03:19.498239 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:19.498218 2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" Jun 03 14:03:19.500579 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:19.500556 2565 status_manager.go:895] "Failed to get status for pod" podUID="365c4e39-df7e-437e-a3ba-cc5c2282c650" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" err="pods \"kuadrant-operator-controller-manager-84b657d985-ctzrl\" is forbidden: User \"system:node:ip-10-0-134-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-89.ec2.internal' and this object" Jun 03 14:03:19.671105 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:19.671023 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgkwc\" (UniqueName: \"kubernetes.io/projected/365c4e39-df7e-437e-a3ba-cc5c2282c650-kube-api-access-vgkwc\") pod \"365c4e39-df7e-437e-a3ba-cc5c2282c650\" (UID: \"365c4e39-df7e-437e-a3ba-cc5c2282c650\") " Jun 03 14:03:19.671105 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:19.671063 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/365c4e39-df7e-437e-a3ba-cc5c2282c650-extensions-socket-volume\") pod \"365c4e39-df7e-437e-a3ba-cc5c2282c650\" (UID: \"365c4e39-df7e-437e-a3ba-cc5c2282c650\") " Jun 03 14:03:19.671354 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:19.671311 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/365c4e39-df7e-437e-a3ba-cc5c2282c650-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "365c4e39-df7e-437e-a3ba-cc5c2282c650" (UID: "365c4e39-df7e-437e-a3ba-cc5c2282c650"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 14:03:19.673333 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:19.673310 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/365c4e39-df7e-437e-a3ba-cc5c2282c650-kube-api-access-vgkwc" (OuterVolumeSpecName: "kube-api-access-vgkwc") pod "365c4e39-df7e-437e-a3ba-cc5c2282c650" (UID: "365c4e39-df7e-437e-a3ba-cc5c2282c650"). InnerVolumeSpecName "kube-api-access-vgkwc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 14:03:19.772510 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:19.772463 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgkwc\" (UniqueName: \"kubernetes.io/projected/365c4e39-df7e-437e-a3ba-cc5c2282c650-kube-api-access-vgkwc\") on node \"ip-10-0-134-89.ec2.internal\" DevicePath \"\"" Jun 03 14:03:19.772510 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:19.772506 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/365c4e39-df7e-437e-a3ba-cc5c2282c650-extensions-socket-volume\") on node \"ip-10-0-134-89.ec2.internal\" DevicePath \"\"" Jun 03 14:03:20.268110 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:20.268077 2565 generic.go:358] "Generic (PLEG): container finished" podID="365c4e39-df7e-437e-a3ba-cc5c2282c650" containerID="a3e51b55337bad11367163089026698eddfa13b8cefc5b9280b58c015258f341" exitCode=2 Jun 03 14:03:20.268556 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:20.268128 2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" Jun 03 14:03:20.268556 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:20.268165 2565 scope.go:117] "RemoveContainer" containerID="a3e51b55337bad11367163089026698eddfa13b8cefc5b9280b58c015258f341" Jun 03 14:03:20.270698 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:20.270670 2565 status_manager.go:895] "Failed to get status for pod" podUID="365c4e39-df7e-437e-a3ba-cc5c2282c650" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" err="pods \"kuadrant-operator-controller-manager-84b657d985-ctzrl\" is forbidden: User \"system:node:ip-10-0-134-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-89.ec2.internal' and this object" Jun 03 14:03:20.277361 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:20.277325 2565 scope.go:117] "RemoveContainer" containerID="a3e51b55337bad11367163089026698eddfa13b8cefc5b9280b58c015258f341" Jun 03 14:03:20.277615 ip-10-0-134-89 kubenswrapper[2565]: E0603 14:03:20.277595 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3e51b55337bad11367163089026698eddfa13b8cefc5b9280b58c015258f341\": container with ID starting with a3e51b55337bad11367163089026698eddfa13b8cefc5b9280b58c015258f341 not found: ID does not exist" containerID="a3e51b55337bad11367163089026698eddfa13b8cefc5b9280b58c015258f341" Jun 03 14:03:20.277693 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:20.277628 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3e51b55337bad11367163089026698eddfa13b8cefc5b9280b58c015258f341"} err="failed to get container status \"a3e51b55337bad11367163089026698eddfa13b8cefc5b9280b58c015258f341\": rpc error: code = NotFound desc = could not find container \"a3e51b55337bad11367163089026698eddfa13b8cefc5b9280b58c015258f341\": container with ID starting with a3e51b55337bad11367163089026698eddfa13b8cefc5b9280b58c015258f341 not found: ID does not exist" Jun 03 14:03:20.278140 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:20.278120 2565 status_manager.go:895] "Failed to get status for pod" podUID="365c4e39-df7e-437e-a3ba-cc5c2282c650" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-ctzrl" err="pods \"kuadrant-operator-controller-manager-84b657d985-ctzrl\" is forbidden: User \"system:node:ip-10-0-134-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-89.ec2.internal' and this object" Jun 03 14:03:20.623937 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:20.623860 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="365c4e39-df7e-437e-a3ba-cc5c2282c650" path="/var/lib/kubelet/pods/365c4e39-df7e-437e-a3ba-cc5c2282c650/volumes" Jun 03 14:03:28.261965 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:03:28.261926 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-l2x47" Jun 03 14:04:50.046282 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.046245 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-ccmjk"] Jun 03 14:04:50.048712 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.046562 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="365c4e39-df7e-437e-a3ba-cc5c2282c650" containerName="manager" Jun 03 14:04:50.048712 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.046573 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="365c4e39-df7e-437e-a3ba-cc5c2282c650" containerName="manager" Jun 03 14:04:50.048712 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.046636 2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="365c4e39-df7e-437e-a3ba-cc5c2282c650" containerName="manager" Jun 03 14:04:50.049550 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.049528 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-7864d68cbd-ccmjk" Jun 03 14:04:50.052065 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.052047 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 03 14:04:50.052747 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.052733 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 03 14:04:50.052809 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.052749 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-bdp9s\"" Jun 03 14:04:50.062592 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.062573 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-ccmjk"] Jun 03 14:04:50.085078 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.085056 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26xr4\" (UniqueName: \"kubernetes.io/projected/d7b230c2-48bb-4e20-a69d-383e4b92500b-kube-api-access-26xr4\") pod \"keycloak-operator-7864d68cbd-ccmjk\" (UID: \"d7b230c2-48bb-4e20-a69d-383e4b92500b\") " pod="keycloak-system/keycloak-operator-7864d68cbd-ccmjk" Jun 03 14:04:50.186486 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.186454 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-26xr4\" (UniqueName: \"kubernetes.io/projected/d7b230c2-48bb-4e20-a69d-383e4b92500b-kube-api-access-26xr4\") pod \"keycloak-operator-7864d68cbd-ccmjk\" (UID: \"d7b230c2-48bb-4e20-a69d-383e4b92500b\") " pod="keycloak-system/keycloak-operator-7864d68cbd-ccmjk" Jun 03 14:04:50.194924 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.194892 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26xr4\" (UniqueName: \"kubernetes.io/projected/d7b230c2-48bb-4e20-a69d-383e4b92500b-kube-api-access-26xr4\") pod \"keycloak-operator-7864d68cbd-ccmjk\" (UID: \"d7b230c2-48bb-4e20-a69d-383e4b92500b\") " pod="keycloak-system/keycloak-operator-7864d68cbd-ccmjk" Jun 03 14:04:50.358957 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.358878 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-7864d68cbd-ccmjk" Jun 03 14:04:50.479671 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.479638 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-7864d68cbd-ccmjk"] Jun 03 14:04:50.482884 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:04:50.482856 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7b230c2_48bb_4e20_a69d_383e4b92500b.slice/crio-662dc8a5d9df8074c554c7a37a67d5b87193b8d478a3e3765605953de6f1ae32 WatchSource:0}: Error finding container 662dc8a5d9df8074c554c7a37a67d5b87193b8d478a3e3765605953de6f1ae32: Status 404 returned error can't find the container with id 662dc8a5d9df8074c554c7a37a67d5b87193b8d478a3e3765605953de6f1ae32 Jun 03 14:04:50.573416 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:50.573381 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-7864d68cbd-ccmjk" event={"ID":"d7b230c2-48bb-4e20-a69d-383e4b92500b","Type":"ContainerStarted","Data":"662dc8a5d9df8074c554c7a37a67d5b87193b8d478a3e3765605953de6f1ae32"} Jun 03 14:04:57.602254 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:57.602209 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-7864d68cbd-ccmjk" event={"ID":"d7b230c2-48bb-4e20-a69d-383e4b92500b","Type":"ContainerStarted","Data":"bc4b264befa28934fb320336eb3d2625b4db501d09c802bb8f38a2e0a2875c44"} Jun 03 14:04:57.619263 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:04:57.619212 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-7864d68cbd-ccmjk" podStartSLOduration=1.554495867 podStartE2EDuration="7.619199811s" podCreationTimestamp="2026-06-03 14:04:50 +0000 UTC" firstStartedPulling="2026-06-03 14:04:50.484695518 +0000 UTC m=+666.429339451" lastFinishedPulling="2026-06-03 14:04:56.549399459 +0000 UTC m=+672.494043395" observedRunningTime="2026-06-03 14:04:57.617640247 +0000 UTC m=+673.562284203" watchObservedRunningTime="2026-06-03 14:04:57.619199811 +0000 UTC m=+673.563843766" Jun 03 14:08:08.682993 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.682957 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz"] Jun 03 14:08:08.686722 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.686700 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.689522 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.689499 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 03 14:08:08.690302 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.690280 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-64tsr\"" Jun 03 14:08:08.690427 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.690306 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 03 14:08:08.690545 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.690527 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 03 14:08:08.700570 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.700541 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz"] Jun 03 14:08:08.712243 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.712218 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.712376 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.712284 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.712376 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.712307 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.712376 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.712325 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7p2l\" (UniqueName: \"kubernetes.io/projected/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-kube-api-access-z7p2l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.712519 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.712493 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.712573 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.712556 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.813139 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.813106 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.813139 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.813145 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.813420 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.813177 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.813420 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.813210 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.813420 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.813226 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.813420 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.813241 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z7p2l\" (UniqueName: \"kubernetes.io/projected/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-kube-api-access-z7p2l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.813705 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.813570 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.813705 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.813584 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.813705 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.813624 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.815562 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.815540 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.815831 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.815810 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:08.822935 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:08.822913 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7p2l\" (UniqueName: \"kubernetes.io/projected/1753217b-2fec-4965-b2fb-cc0f7d6c82b7-kube-api-access-z7p2l\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz\" (UID: \"1753217b-2fec-4965-b2fb-cc0f7d6c82b7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:09.000104 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:09.000066 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" Jun 03 14:08:09.138920 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:09.138880 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz"] Jun 03 14:08:09.141151 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:08:09.141121 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1753217b_2fec_4965_b2fb_cc0f7d6c82b7.slice/crio-56107bb16ce8ec8723599625645d3b7dd687cf132fd720a4f394054bb561961d WatchSource:0}: Error finding container 56107bb16ce8ec8723599625645d3b7dd687cf132fd720a4f394054bb561961d: Status 404 returned error can't find the container with id 56107bb16ce8ec8723599625645d3b7dd687cf132fd720a4f394054bb561961d Jun 03 14:08:09.142927 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:09.142908 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 14:08:09.218783 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:09.218748 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" event={"ID":"1753217b-2fec-4965-b2fb-cc0f7d6c82b7","Type":"ContainerStarted","Data":"56107bb16ce8ec8723599625645d3b7dd687cf132fd720a4f394054bb561961d"} Jun 03 14:08:13.235179 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:13.235141 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz" event={"ID":"1753217b-2fec-4965-b2fb-cc0f7d6c82b7","Type":"ContainerStarted","Data":"cd152037070f6b588fa7d175030ff3c85597b53cd352946da2dc5e96d893fe0a"} Jun 03 14:08:29.598747 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.598714 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw"] Jun 03 14:08:29.603764 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.603744 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.606179 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.606161 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 03 14:08:29.611959 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.611937 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw"] Jun 03 14:08:29.696591 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.696563 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1118ae80-29dd-448d-8b13-8bf3bfaf013a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.696754 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.696598 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1118ae80-29dd-448d-8b13-8bf3bfaf013a-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.696754 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.696686 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gbrg\" (UniqueName: \"kubernetes.io/projected/1118ae80-29dd-448d-8b13-8bf3bfaf013a-kube-api-access-9gbrg\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.696754 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.696718 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1118ae80-29dd-448d-8b13-8bf3bfaf013a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.696754 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.696740 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1118ae80-29dd-448d-8b13-8bf3bfaf013a-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.696991 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.696768 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1118ae80-29dd-448d-8b13-8bf3bfaf013a-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.798323 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.798282 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1118ae80-29dd-448d-8b13-8bf3bfaf013a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.798493 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.798334 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1118ae80-29dd-448d-8b13-8bf3bfaf013a-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.798493 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.798423 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9gbrg\" (UniqueName: \"kubernetes.io/projected/1118ae80-29dd-448d-8b13-8bf3bfaf013a-kube-api-access-9gbrg\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.798493 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.798453 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1118ae80-29dd-448d-8b13-8bf3bfaf013a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.798493 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.798483 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1118ae80-29dd-448d-8b13-8bf3bfaf013a-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.798691 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.798516 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1118ae80-29dd-448d-8b13-8bf3bfaf013a-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.798744 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.798712 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1118ae80-29dd-448d-8b13-8bf3bfaf013a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.798834 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.798809 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1118ae80-29dd-448d-8b13-8bf3bfaf013a-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.798902 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.798883 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1118ae80-29dd-448d-8b13-8bf3bfaf013a-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.800823 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.800800 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1118ae80-29dd-448d-8b13-8bf3bfaf013a-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.801223 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.801200 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1118ae80-29dd-448d-8b13-8bf3bfaf013a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.812503 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.812479 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gbrg\" (UniqueName: \"kubernetes.io/projected/1118ae80-29dd-448d-8b13-8bf3bfaf013a-kube-api-access-9gbrg\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw\" (UID: \"1118ae80-29dd-448d-8b13-8bf3bfaf013a\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:29.914947 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:29.914872 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:30.047917 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:30.047878 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw"] Jun 03 14:08:30.050090 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:08:30.050056 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1118ae80_29dd_448d_8b13_8bf3bfaf013a.slice/crio-70c2b9aa3da0d42d4cc0ea2c4bed348a8f253d48b7e17bfc12223718c4c6c519 WatchSource:0}: Error finding container 70c2b9aa3da0d42d4cc0ea2c4bed348a8f253d48b7e17bfc12223718c4c6c519: Status 404 returned error can't find the container with id 70c2b9aa3da0d42d4cc0ea2c4bed348a8f253d48b7e17bfc12223718c4c6c519 Jun 03 14:08:30.295553 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:30.295518 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" event={"ID":"1118ae80-29dd-448d-8b13-8bf3bfaf013a","Type":"ContainerStarted","Data":"e20625d99ed0155f507e0d01e9936f639eff4d5ff52aa0bb8883fbd6133420b5"} Jun 03 14:08:30.295553 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:30.295555 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" event={"ID":"1118ae80-29dd-448d-8b13-8bf3bfaf013a","Type":"ContainerStarted","Data":"70c2b9aa3da0d42d4cc0ea2c4bed348a8f253d48b7e17bfc12223718c4c6c519"} Jun 03 14:08:33.306537 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:33.306500 2565 generic.go:358] "Generic (PLEG): container finished" podID="1118ae80-29dd-448d-8b13-8bf3bfaf013a" containerID="e20625d99ed0155f507e0d01e9936f639eff4d5ff52aa0bb8883fbd6133420b5" exitCode=0 Jun 03 14:08:33.306984 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:33.306583 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" event={"ID":"1118ae80-29dd-448d-8b13-8bf3bfaf013a","Type":"ContainerDied","Data":"e20625d99ed0155f507e0d01e9936f639eff4d5ff52aa0bb8883fbd6133420b5"} Jun 03 14:08:42.335953 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:42.335917 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" event={"ID":"1118ae80-29dd-448d-8b13-8bf3bfaf013a","Type":"ContainerStarted","Data":"8447b4a8f25d092b0b6e1ec74ce53cb036bd45b7ffcd472cb7055704d6578796"} Jun 03 14:08:42.356066 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:42.356017 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" podStartSLOduration=4.959347434 podStartE2EDuration="13.356004064s" podCreationTimestamp="2026-06-03 14:08:29 +0000 UTC" firstStartedPulling="2026-06-03 14:08:33.307282912 +0000 UTC m=+889.251926845" lastFinishedPulling="2026-06-03 14:08:41.703939538 +0000 UTC m=+897.648583475" observedRunningTime="2026-06-03 14:08:42.355067468 +0000 UTC m=+898.299711422" watchObservedRunningTime="2026-06-03 14:08:42.356004064 +0000 UTC m=+898.300648019" Jun 03 14:08:49.915885 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:49.915847 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:49.915885 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:49.915888 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:49.928459 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:49.928429 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:08:50.371610 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:08:50.371578 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw" Jun 03 14:15:25.794875 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:25.794846 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-549d6557c8-fkfmq_7fe4b8fe-65f0-4b05-8690-d2469d0651d7/nginx/0.log" Jun 03 14:15:25.903802 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:25.903775 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-x2998_ae7117df-a58a-4603-9479-22064dc28838/manager/0.log" Jun 03 14:15:26.013534 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:26.013501 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-7txxr_0c76e725-da9b-4032-a218-f723626f5402/manager/0.log" Jun 03 14:15:26.472414 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:26.472380 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-6qwz7_3d24095a-9983-4de2-9130-e6db34032792/server/0.log" Jun 03 14:15:26.582749 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:26.582721 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-8mtfw_b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f/manager/0.log" Jun 03 14:15:26.973415 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:26.973390 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c68786f4b-nz2pp_5989b631-fc30-42c5-9d33-ce116e0a33a8/copy-manifests/0.log" Jun 03 14:15:26.983614 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:26.983584 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c68786f4b-nz2pp_5989b631-fc30-42c5-9d33-ce116e0a33a8/manager/0.log" Jun 03 14:15:29.094761 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:29.094730 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-l2x47_38d10a07-2d15-4089-9346-39afc5d371f7/manager/0.log" Jun 03 14:15:29.542907 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:29.542880 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-km2pw_a74827eb-4b95-4cf6-9f34-fc17310ddf7d/discovery/0.log" Jun 03 14:15:29.651033 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:29.651004 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-655d7d459-5lvfh_b06dd9c5-058e-4bb8-913d-481be4b8e92d/kube-auth-proxy/0.log" Jun 03 14:15:30.765583 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:30.765553 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f994rbz_1753217b-2fec-4965-b2fb-cc0f7d6c82b7/storage-initializer/0.log" Jun 03 14:15:30.883285 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:30.883259 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw_1118ae80-29dd-448d-8b13-8bf3bfaf013a/storage-initializer/0.log" Jun 03 14:15:30.896285 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:30.896249 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-pfbsw_1118ae80-29dd-448d-8b13-8bf3bfaf013a/main/0.log" Jun 03 14:15:38.739202 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:38.739171 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nsnhl_561a2ef6-f963-4946-bdf9-8336eaae20d6/global-pull-secret-syncer/0.log" Jun 03 14:15:38.904756 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:38.904726 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-np4q9_77b40cee-f20f-4399-b4ff-42fc5f16335e/konnectivity-agent/0.log" Jun 03 14:15:38.995153 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:38.995069 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-89.ec2.internal_92c7e5b782277de57331bd32a595193d/haproxy/0.log" Jun 03 14:15:43.386435 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:43.386399 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-l2x47_38d10a07-2d15-4089-9346-39afc5d371f7/manager/0.log" Jun 03 14:15:45.183639 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:45.183605 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5wd67_f30986ba-b02c-4645-be24-c5867f3a7f03/node-exporter/0.log" Jun 03 14:15:45.224593 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:45.224566 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5wd67_f30986ba-b02c-4645-be24-c5867f3a7f03/kube-rbac-proxy/0.log" Jun 03 14:15:45.253135 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:45.253113 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5wd67_f30986ba-b02c-4645-be24-c5867f3a7f03/init-textfile/0.log" Jun 03 14:15:47.323528 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:47.323497 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-9qlnz_4dfd6ead-68a6-493b-9ad0-37c7c81aaf9b/networking-console-plugin/0.log" Jun 03 14:15:48.095133 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.095096 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf"] Jun 03 14:15:48.098729 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.098707 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.101122 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.101101 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c6tt9\"/\"kube-root-ca.crt\"" Jun 03 14:15:48.101931 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.101909 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c6tt9\"/\"openshift-service-ca.crt\"" Jun 03 14:15:48.102106 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.101920 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c6tt9\"/\"default-dockercfg-d9dxq\"" Jun 03 14:15:48.107502 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.107482 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf"] Jun 03 14:15:48.210115 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.210080 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/21324861-eaf2-4ee2-8505-efb2902e2bf8-podres\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.210286 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.210139 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/21324861-eaf2-4ee2-8505-efb2902e2bf8-proc\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.210286 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.210190 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21324861-eaf2-4ee2-8505-efb2902e2bf8-lib-modules\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.210286 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.210224 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21324861-eaf2-4ee2-8505-efb2902e2bf8-sys\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.210446 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.210301 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7l4r\" (UniqueName: \"kubernetes.io/projected/21324861-eaf2-4ee2-8505-efb2902e2bf8-kube-api-access-c7l4r\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.311247 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.311214 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/21324861-eaf2-4ee2-8505-efb2902e2bf8-proc\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.311247 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.311254 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21324861-eaf2-4ee2-8505-efb2902e2bf8-lib-modules\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.311511 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.311283 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21324861-eaf2-4ee2-8505-efb2902e2bf8-sys\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.311511 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.311329 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/21324861-eaf2-4ee2-8505-efb2902e2bf8-proc\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.311511 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.311385 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c7l4r\" (UniqueName: \"kubernetes.io/projected/21324861-eaf2-4ee2-8505-efb2902e2bf8-kube-api-access-c7l4r\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.311511 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.311413 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/21324861-eaf2-4ee2-8505-efb2902e2bf8-podres\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.311511 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.311425 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21324861-eaf2-4ee2-8505-efb2902e2bf8-sys\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.311511 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.311438 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21324861-eaf2-4ee2-8505-efb2902e2bf8-lib-modules\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.311695 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.311591 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/21324861-eaf2-4ee2-8505-efb2902e2bf8-podres\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.320010 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.319979 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7l4r\" (UniqueName: \"kubernetes.io/projected/21324861-eaf2-4ee2-8505-efb2902e2bf8-kube-api-access-c7l4r\") pod \"perf-node-gather-daemonset-472mf\" (UID: \"21324861-eaf2-4ee2-8505-efb2902e2bf8\") " pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.408684 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.408609 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.536993 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.536965 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf"] Jun 03 14:15:48.539038 ip-10-0-134-89 kubenswrapper[2565]: W0603 14:15:48.539006 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod21324861_eaf2_4ee2_8505_efb2902e2bf8.slice/crio-fe5d8fe630a69b2115846c80f81f4774bca327356132881bfaf54b13fdae54ad WatchSource:0}: Error finding container fe5d8fe630a69b2115846c80f81f4774bca327356132881bfaf54b13fdae54ad: Status 404 returned error can't find the container with id fe5d8fe630a69b2115846c80f81f4774bca327356132881bfaf54b13fdae54ad Jun 03 14:15:48.540651 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.540631 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 14:15:48.757315 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.757280 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" event={"ID":"21324861-eaf2-4ee2-8505-efb2902e2bf8","Type":"ContainerStarted","Data":"73c09cf5a4488cf6fc48316a56deeb08b50832b0183412b72902ae276249a405"} Jun 03 14:15:48.757500 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.757322 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" event={"ID":"21324861-eaf2-4ee2-8505-efb2902e2bf8","Type":"ContainerStarted","Data":"fe5d8fe630a69b2115846c80f81f4774bca327356132881bfaf54b13fdae54ad"} Jun 03 14:15:48.757500 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.757387 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:48.775452 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:48.775402 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" podStartSLOduration=0.775387396 podStartE2EDuration="775.387396ms" podCreationTimestamp="2026-06-03 14:15:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 14:15:48.773667983 +0000 UTC m=+1324.718311948" watchObservedRunningTime="2026-06-03 14:15:48.775387396 +0000 UTC m=+1324.720031350" Jun 03 14:15:49.710275 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:49.710242 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7d2wt_6724d1a1-68ea-4f1d-a782-7bbaf376213c/dns/0.log" Jun 03 14:15:49.733222 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:49.733193 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7d2wt_6724d1a1-68ea-4f1d-a782-7bbaf376213c/kube-rbac-proxy/0.log" Jun 03 14:15:49.852494 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:49.852454 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gbtxm_8c5d4af8-6b15-491f-9c16-884f366947e3/dns-node-resolver/0.log" Jun 03 14:15:50.399464 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:50.399435 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6ns2r_dedc278e-adec-4d58-bafb-047e9a36c84c/node-ca/0.log" Jun 03 14:15:51.340472 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:51.340436 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-km2pw_a74827eb-4b95-4cf6-9f34-fc17310ddf7d/discovery/0.log" Jun 03 14:15:51.359595 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:51.359568 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-655d7d459-5lvfh_b06dd9c5-058e-4bb8-913d-481be4b8e92d/kube-auth-proxy/0.log" Jun 03 14:15:51.996107 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:51.996078 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p66h6_04c03b8a-4b1b-4b45-8f1d-cc87a84f87ea/serve-healthcheck-canary/0.log" Jun 03 14:15:52.561184 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:52.561157 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4jdkm_49e68b9c-5f36-4ff4-a96e-550613b82dc3/kube-rbac-proxy/0.log" Jun 03 14:15:52.586372 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:52.586331 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4jdkm_49e68b9c-5f36-4ff4-a96e-550613b82dc3/exporter/0.log" Jun 03 14:15:52.623539 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:52.623495 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4jdkm_49e68b9c-5f36-4ff4-a96e-550613b82dc3/extractor/0.log" Jun 03 14:15:54.770716 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:54.770690 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c6tt9/perf-node-gather-daemonset-472mf" Jun 03 14:15:54.957296 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:54.957266 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-549d6557c8-fkfmq_7fe4b8fe-65f0-4b05-8690-d2469d0651d7/nginx/0.log" Jun 03 14:15:54.980795 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:54.980774 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-x2998_ae7117df-a58a-4603-9479-22064dc28838/manager/0.log" Jun 03 14:15:55.014082 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:55.014049 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-7txxr_0c76e725-da9b-4032-a218-f723626f5402/manager/0.log" Jun 03 14:15:55.116824 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:55.116744 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-6577596cd5-6qwz7_3d24095a-9983-4de2-9130-e6db34032792/server/0.log" Jun 03 14:15:55.162900 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:55.162874 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-8mtfw_b135e85a-2c63-4a1e-a8d2-bdd3c29d9b9f/manager/0.log" Jun 03 14:15:55.357036 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:55.357008 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c68786f4b-nz2pp_5989b631-fc30-42c5-9d33-ce116e0a33a8/manager/0.log" Jun 03 14:15:55.410690 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:55.410610 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c68786f4b-nz2pp_5989b631-fc30-42c5-9d33-ce116e0a33a8/copy-manifests/0.log" Jun 03 14:15:56.902935 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:15:56.902867 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-7z2c9_70a7f4a1-a8c1-4640-9541-48d929750adf/openshift-lws-operator/0.log" Jun 03 14:16:02.942302 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:16:02.942225 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f5chv_1d997ce2-6758-4937-8e63-dad6bf7b689a/kube-multus-additional-cni-plugins/0.log" Jun 03 14:16:02.963199 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:16:02.963171 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f5chv_1d997ce2-6758-4937-8e63-dad6bf7b689a/egress-router-binary-copy/0.log" Jun 03 14:16:02.983108 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:16:02.983089 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f5chv_1d997ce2-6758-4937-8e63-dad6bf7b689a/cni-plugins/0.log" Jun 03 14:16:03.006322 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:16:03.006304 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f5chv_1d997ce2-6758-4937-8e63-dad6bf7b689a/bond-cni-plugin/0.log" Jun 03 14:16:03.029180 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:16:03.029162 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f5chv_1d997ce2-6758-4937-8e63-dad6bf7b689a/routeoverride-cni/0.log" Jun 03 14:16:03.068621 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:16:03.068599 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f5chv_1d997ce2-6758-4937-8e63-dad6bf7b689a/whereabouts-cni-bincopy/0.log" Jun 03 14:16:03.090022 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:16:03.089995 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f5chv_1d997ce2-6758-4937-8e63-dad6bf7b689a/whereabouts-cni/0.log" Jun 03 14:16:03.477581 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:16:03.477498 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zp6hm_4df1db34-2741-48c1-844e-231fbafcd068/kube-multus/0.log" Jun 03 14:16:03.498204 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:16:03.498153 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5m5l8_277e2d8b-f5ab-4f08-bb7e-b6981b3932fc/network-metrics-daemon/0.log" Jun 03 14:16:03.519677 ip-10-0-134-89 kubenswrapper[2565]: I0603 14:16:03.519601 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5m5l8_277e2d8b-f5ab-4f08-bb7e-b6981b3932fc/kube-rbac-proxy/0.log"